I have a Numpy array of shape (5,5,3,2). I want to take the element (1,4) of that matrix, which is also a matrix of shape (3,2), and add an element to it -so it becomes a (4,2) array.
The code I'm using is the following:
import numpy as np
a = np.random.rand(5,5,3,2)
a = np.array(a, dtype = object) #So I can have different size sub-matrices
a[2][3] = np.append(a[2][3],[[1.0,1.0]],axis=0) #a[2][3] shape = (3,2)
I'm always obtaining the error:
ValueError: could not broadcast input array from shape (4,2) into shape (3,2)
I understand that the shape returned by the np.append function is not the same as the a[2][3] sub-array, but I thought that the dtype=object would solve my problem. However, I need to do this. Is there any way to go around this limitation?
I also tried to use the insert function but I don't know how could I add the element in the place I want.
Make sure you understand what you have produced. That requires checking the shape and dtype, and possibly looking at the values
In [29]: a = np.random.rand(5,5,3,2)
In [30]: b=np.array(a, dtype=object)
In [31]: a.shape
Out[31]: (5, 5, 3, 2) # a is a 4d array
In [32]: a.dtype
Out[32]: dtype('float64')
In [33]: b.shape
Out[33]: (5, 5, 3, 2) # so is b
In [34]: b.dtype
Out[34]: dtype('O')
In [35]: b[2,3].shape
Out[35]: (3, 2)
In [36]: c=np.append(b[2,3],[[1,1]],axis=0)
In [37]: c.shape
Out[37]: (4, 2)
In [38]: c.dtype
Out[38]: dtype('O')
b[2][3] is also an array. b[2,3] is the proper numpy way of indexing 2 dimensions.
I suspect you wanted b to be a (5,5) array containing arrays (as objects), and you think that you you can simply replace one of those with a (4,2) array. But the b constructor simply changes the floats of a to objects, without changing the shape (or 4d nature) of b.
I could construct a (5,5) object array, and fill it with values from a. And then replace one of those values with a (4,2) array:
In [39]: B=np.empty((5,5),dtype=object)
In [40]: for i in range(5):
...: for j in range(5):
...: B[i,j]=a[i,j,:,:]
...:
In [41]: B.shape
Out[41]: (5, 5)
In [42]: B.dtype
Out[42]: dtype('O')
In [43]: B[2,3]
Out[43]:
array([[ 0.03827568, 0.63411023],
[ 0.28938383, 0.7951006 ],
[ 0.12217603, 0.304537 ]])
In [44]: B[2,3]=c
In [46]: B[2,3].shape
Out[46]: (4, 2)
This constructor for B is a bit crude. I've answered other questions about creating/filling object arrays, but I'm not going to take the time here to streamline this case. It's for illustration purposes only.
In an array of object, any element can be indeed an array (or any kind of object).
import numpy as np
a = np.random.rand(5,5,3,2)
a = np.array(a, dtype=object)
# Assign an 1D array to the array element ``a[2][3][0][0]``:
a[2][3][0][0] = np.arange(10)
a[2][3][0][0][9] # 9
However a[2][3] is not an array element, it is a whole array.
a[2][3].ndim # 2
Therefore when you do a[2][3] = (something) you are using broadcasting instead of assigning an element: numpy tries to replace the content of the subarray a[2][3] and fails because of shape mismatch. The memory layout of numpy arrays does not allow to change the shape of subarrays.
Edit: Instead of using numpy arrays you could use nested lists. These nested lists can have arbitrary sizes. Note that the memory is higher and that the access time is higher compared to numpy array.
import numpy as np
a = np.random.rand(5,5,3,2)
a = np.array(a, dtype=object)
b = np.append(a[2][3], [[1.0,1.0]],axis=0)
a_list = a.tolist()
a_list[2][3] = b.tolist()
The problem here, is that you try to assign to a[2][3]
Make a new array instead.
new_array = np.append(a[2][3],np.array([[1.0,1.0]]),axis=0)
Related
I use Python and NumPy and have some problems with "transpose":
import numpy as np
a = np.array([5,4])
print(a)
print(a.T)
Invoking a.T is not transposing the array. If a is for example [[],[]] then it transposes correctly, but I need the transpose of [...,...,...].
It's working exactly as it's supposed to. The transpose of a 1D array is still a 1D array! (If you're used to matlab, it fundamentally doesn't have a concept of a 1D array. Matlab's "1D" arrays are 2D.)
If you want to turn your 1D vector into a 2D array and then transpose it, just slice it with np.newaxis (or None, they're the same, newaxis is just more readable).
import numpy as np
a = np.array([5,4])[np.newaxis]
print(a)
print(a.T)
Generally speaking though, you don't ever need to worry about this. Adding the extra dimension is usually not what you want, if you're just doing it out of habit. Numpy will automatically broadcast a 1D array when doing various calculations. There's usually no need to distinguish between a row vector and a column vector (neither of which are vectors. They're both 2D!) when you just want a vector.
Use two bracket pairs instead of one. This creates a 2D array, which can be transposed, unlike the 1D array you create if you use one bracket pair.
import numpy as np
a = np.array([[5, 4]])
a.T
More thorough example:
>>> a = [3,6,9]
>>> b = np.array(a)
>>> b.T
array([3, 6, 9]) #Here it didn't transpose because 'a' is 1 dimensional
>>> b = np.array([a])
>>> b.T
array([[3], #Here it did transpose because a is 2 dimensional
[6],
[9]])
Use numpy's shape method to see what is going on here:
>>> b = np.array([10,20,30])
>>> b.shape
(3,)
>>> b = np.array([[10,20,30]])
>>> b.shape
(1, 3)
For 1D arrays:
a = np.array([1, 2, 3, 4])
a = a.reshape((-1, 1)) # <--- THIS IS IT
print a
array([[1],
[2],
[3],
[4]])
Once you understand that -1 here means "as many rows as needed", I find this to be the most readable way of "transposing" an array. If your array is of higher dimensionality simply use a.T.
You can convert an existing vector into a matrix by wrapping it in an extra set of square brackets...
from numpy import *
v=array([5,4]) ## create a numpy vector
array([v]).T ## transpose a vector into a matrix
numpy also has a matrix class (see array vs. matrix)...
matrix(v).T ## transpose a vector into a matrix
numpy 1D array --> column/row matrix:
>>> a=np.array([1,2,4])
>>> a[:, None] # col
array([[1],
[2],
[4]])
>>> a[None, :] # row, or faster `a[None]`
array([[1, 2, 4]])
And as #joe-kington said, you can replace None with np.newaxis for readability.
To 'transpose' a 1d array to a 2d column, you can use numpy.vstack:
>>> numpy.vstack(numpy.array([1,2,3]))
array([[1],
[2],
[3]])
It also works for vanilla lists:
>>> numpy.vstack([1,2,3])
array([[1],
[2],
[3]])
instead use arr[:,None] to create column vector
You can only transpose a 2D array. You can use numpy.matrix to create a 2D array. This is three years late, but I am just adding to the possible set of solutions:
import numpy as np
m = np.matrix([2, 3])
m.T
Basically what the transpose function does is to swap the shape and strides of the array:
>>> a = np.ones((1,2,3))
>>> a.shape
(1, 2, 3)
>>> a.T.shape
(3, 2, 1)
>>> a.strides
(48, 24, 8)
>>> a.T.strides
(8, 24, 48)
In case of 1D numpy array (rank-1 array) the shape and strides are 1-element tuples and cannot be swapped, and the transpose of such an 1D array returns it unchanged. Instead, you can transpose a "row-vector" (numpy array of shape (1, n)) into a "column-vector" (numpy array of shape (n, 1)). To achieve this you have to first convert your 1D numpy array into row-vector and then swap the shape and strides (transpose it). Below is a function that does it:
from numpy.lib.stride_tricks import as_strided
def transpose(a):
a = np.atleast_2d(a)
return as_strided(a, shape=a.shape[::-1], strides=a.strides[::-1])
Example:
>>> a = np.arange(3)
>>> a
array([0, 1, 2])
>>> transpose(a)
array([[0],
[1],
[2]])
>>> a = np.arange(1, 7).reshape(2,3)
>>> a
array([[1, 2, 3],
[4, 5, 6]])
>>> transpose(a)
array([[1, 4],
[2, 5],
[3, 6]])
Of course you don't have to do it this way since you have a 1D array and you can directly reshape it into (n, 1) array by a.reshape((-1, 1)) or a[:, None]. I just wanted to demonstrate how transposing an array works.
Another solution.... :-)
import numpy as np
a = [1,2,4]
[1, 2, 4]
b = np.array([a]).T
array([[1],
[2],
[4]])
The name of the function in numpy is column_stack.
>>>a=np.array([5,4])
>>>np.column_stack(a)
array([[5, 4]])
I am just consolidating the above post, hope it will help others to save some time:
The below array has (2, )dimension, it's a 1-D array,
b_new = np.array([2j, 3j])
There are two ways to transpose a 1-D array:
slice it with "np.newaxis" or none.!
print(b_new[np.newaxis].T.shape)
print(b_new[None].T.shape)
other way of writing, the above without T operation.!
print(b_new[:, np.newaxis].shape)
print(b_new[:, None].shape)
Wrapping [ ] or using np.matrix, means adding a new dimension.!
print(np.array([b_new]).T.shape)
print(np.matrix(b_new).T.shape)
There is a method not described in the answers but described in the documentation for the numpy.ndarray.transpose method:
For a 1-D array this has no effect, as a transposed vector is simply the same vector. To convert a 1-D array into a 2D column vector, an additional dimension must be added. np.atleast2d(a).T achieves this, as does a[:, np.newaxis].
One can do:
import numpy as np
a = np.array([5,4])
print(a)
print(np.atleast_2d(a).T)
Which (imo) is nicer than using newaxis.
As some of the comments above mentioned, the transpose of 1D arrays are 1D arrays, so one way to transpose a 1D array would be to convert the array to a matrix like so:
np.transpose(a.reshape(len(a), 1))
To transpose a 1-D array (flat array) as you have in your example, you can use the np.expand_dims() function:
>>> a = np.expand_dims(np.array([5, 4]), axis=1)
array([[5],
[4]])
np.expand_dims() will add a dimension to the chosen axis. In this case, we use axis=1, which adds a column dimension, effectively transposing your original flat array.
I have 2 Numpy array <type 'numpy.ndarray'> with shape of (10,) (10, 6) and I would like to concat the first one with the second. The numpy array provided below,
r1
['467c8100-7f13-4244-81ee-5e2a0f8218a8',
'71a4b5b2-80d6-4c12-912f-fc71be8d923e',
'7a3e0168-e47d-4203-98f2-a54a46c62ae0',
'7dfd43e7-ced1-435f-a0f9-80cfd00ae246',
'85dbc70e-c773-43ee-b434-8f458d295d10',
'a56b2bc3-4a81-469e-bc5f-b3aaa520db05',
'a9e8996f-ff35-4bfb-bbd9-ede5ffecd4d8',
'c3037410-0c2e-40f8-a844-ac0664a05783',
'c5618563-10c0-425b-a11b-2fcf931f0ff7',
'f65e6cea-892e-4335-8e86-bf7f083b5f53']
r2
[[1.55000000e+02, 5.74151515e-01, 1.55000000e+02, 5.74151515e-01, 3.49000000e+02, 1.88383585e+00],
[5.00000000e+00, 1.91871554e-01, 1.03000000e+02, 1.22893828e+00, 2.95000000e+02, 3.21148368e+00],
[7.10000000e+01, 1.15231270e-01, 2.42000000e+02, 5.78527276e-01, 4.09000000e+02, 2.67915246e+00],
[3.60000000e+01, 7.10066720e-01, 2.42000000e+02, 1.80213634e+00, 4.12000000e+02, 4.16314391e+00],
[1.15000000e+02, 1.05120284e+00, 1.30000000e+02, 1.71697773e+00, 2.53000000e+02, 2.73640301e+00],
[4.70000000e+01, 2.19434656e-01, 3.23000000e+02, 4.84093786e+00, 5.75000000e+02, 7.00530186e+00],
[5.50000000e+01, 1.22614463e+00, 1.04000000e+02, 1.55392099e+00, 4.34000000e+02, 4.13661261e+00],
[3.90000000e+01, 3.34816889e-02, 1.10000000e+02, 2.54431753e-01, 2.76000000e+02, 1.52322736e+00],
[3.43000000e+02, 2.93550948e+00, 5.84000000e+02, 5.27968165e+00, 7.45000000e+02, 7.57657633e+00],
[1.66000000e+02, 1.01436635e+00, 2.63000000e+02, 2.69197514e+00, 8.13000000e+02, 7.96477735e+00]]
I tried to concatenate with the command np.concatenate((r1, r2)), it returns with the message of ValueError: all the input arrays must have same number of dimensions which I don't understand. Because, the r1 can possibly concat with the r2 and can form a whole new array and make a new array of 10 x 7 as result.
How to solve this problem ?
Numpy offers an easy way to concatenate along the second axis.
np.c_[r2,r1]
You can reshape r1 to make it two-dimensional and specify the axis along which the arrays should be joined:
import numpy as np
r1 = np.ones((10,))
r2 = np.zeros((10, 6))
np.concatenate((r1.reshape(10, 1), r2), axis=1)
These 2 array have a dtype and shape mismatch:
In [174]: r1.shape
Out[174]: (10,)
In [175]: r1.dtype
Out[175]: dtype('<U36')
In [177]: r2.shape
Out[177]: (10, 6)
In [178]: r2.dtype
Out[178]: dtype('float64')
If you add a dimension to r1, so it is now (10,1), you can concatenate on axis=1. But note the dtype - the floats have been turned into strings:
In [181]: r12 =np.concatenate((r1[:,None], r2), axis=1)
In [182]: r12.shape
Out[182]: (10, 7)
In [183]: r12.dtype
Out[183]: dtype('<U36')
In [184]: r12[0,:]
Out[184]:
array(['467c8100-7f13-4244-81ee-5e2a0f8218a8', '155.0', '0.574151515',
'155.0', '0.574151515', '349.0', '1.88383585'],
dtype='<U36')
A way to mix string and floats is with structured array, for example:
In [185]: res=np.zeros((10,),dtype='U36,(6)f')
In [186]: res.dtype
Out[186]: dtype([('f0', '<U36'), ('f1', '<f4', (6,))])
In [187]: res['f0']=r1
In [188]: res['f1']=r2
In [192]: res.shape
Out[192]: (10,)
In [193]: res[0]
Out[193]: ('467c8100-7f13-4244-81ee-5e2a0f8218a8', [ 155. , 0.57415152, 155. , 0.57415152, 349. , 1.88383579])
We could also make a (10,7) array with dtype=object. But most array operations won't work with such a mix of strings and floats. And the ones that work are slower.
Why do you want to concatenate these arrays? What do you intend to do with the result? That dtype mismatch is more serious than the shape mismatch.
import numpy as np
a = np.array([1,2,3,4])
print a.shape[0]
Why it will output 4?
The array [1,2,3,4], it's rows should be 1, I think , so who can explain the reason for me?
because
print(a.shape) # -> (4,)
what you think (or want?) to have is
a = np.array([[1],[2],[3],[4]])
print(a.shape) # -> (4, 1)
or rather (?)
a = np.array([[1, 2 , 3 , 4]])
print(a.shape) # -> (1, 4)
If you'll print a.ndim you'll get 1. That means that a is a one-dimensional array (has rank 1 in numpy terminology), with axis length = 4. It's different from 2D matrix with a single row or column (rank 2).
More on ranks
Related questions:
numpy: 1D array with various shape
Python: Differentiating between row and column vectors
The shape attribute for numpy arrays returns the dimensions of the array. If a has n rows and m columns, then a.shape is (n,m). So a.shape[0] is n and a.shape[1] is m.
numpy arrays returns the dimensions of the array. So, when you create an array using,
a = np.array([1,2,3,4])
you get an array with 4 dimensions. You can check it by printing the shape,
print(a.shape) #(4,)
So, what you get is NOT a 1x4 matrix. If you want that do,
a = numpy.array([1,2,3,4]).reshape((1,4))
print(a.shape)
Or even better,
a = numpy.array([[1,2,3,4]])
a = np.array([1, 2, 3, 4])
by doing this, you get a a as a ndarray, and it is a one-dimension array. Here, the shape (4,) means the array is indexed by a single index which runs from 0 to 3. You can access the elements by the index 0~3. It is different from multi-dimensional arrays.
You can refer to more help from this link Difference between numpy.array shape (R, 1) and (R,).
I want to take an array with shape (N,), and dtype=object, of arrays that all have the same shape, shape, and create an array with shape == (N,) + shape. I was wondering if anyone knew the best way to do this. Here's an example.
import numpy as np
array = np.empty(4, dtype=object)
array[:] = [np.ones([3, 2])]
array = np.array(array.tolist())
print array.dtype
# float64
print array.shape
# (4, 3, 2)
If you already know the shape of your inner arrays (here, (3,2)), you could simplify the whole process as
subshape = (3,2)
a = np.empty(tuple([N,]+list(subshape)), dtype=object)
a[:] = np.ones(subshape)
That will let you avoid unnecessary conversions to/from lists.
Now, assuming you have a (N,) object array a where each element is a subshape float array, you could do:
a = np.vstack(a)
a.shape = [N,] + list(subshape)
or more simply:
a = np.array(a.tolist(), dtype=float)
the .tolist conversion might not be very efficient, though.
How do I get the dimensions of an array? For instance, this is 2x2:
a = np.array([[1,2],[3,4]])
Use .shape to obtain a tuple of array dimensions:
>>> a.shape
(2, 2)
First:
By convention, in Python world, the shortcut for numpy is np, so:
In [1]: import numpy as np
In [2]: a = np.array([[1,2],[3,4]])
Second：
In Numpy, dimension, axis/axes, shape are related and sometimes similar concepts:
dimension
In Mathematics/Physics, dimension or dimensionality is informally defined as the minimum number of coordinates needed to specify any point within a space. But in Numpy, according to the numpy doc, it's the same as axis/axes:
In Numpy dimensions are called axes. The number of axes is rank.
In [3]: a.ndim # num of dimensions/axes, *Mathematics definition of dimension*
Out[3]: 2
axis/axes
the nth coordinate to index an array in Numpy. And multidimensional arrays can have one index per axis.
In [4]: a[1,0] # to index `a`, we specific 1 at the first axis and 0 at the second axis.
Out[4]: 3 # which results in 3 (locate at the row 1 and column 0, 0-based index)
shape
describes how many data (or the range) along each available axis.
In [5]: a.shape
Out[5]: (2, 2) # both the first and second axis have 2 (columns/rows/pages/blocks/...) data
import numpy as np
>>> np.shape(a)
(2,2)
Also works if the input is not a numpy array but a list of lists
>>> a = [[1,2],[1,2]]
>>> np.shape(a)
(2,2)
Or a tuple of tuples
>>> a = ((1,2),(1,2))
>>> np.shape(a)
(2,2)
Use .shape:
In: a = np.array([[1,2,3],[4,5,6]])
In: a.shape
Out: (2, 3)
In: a.shape[0] # x axis
Out: 2
In: a.shape[1] # y axis
Out: 3
You can use .ndim for dimension and .shape to know the exact dimension:
>>> var = np.array([[1,2,3,4,5,6], [1,2,3,4,5,6]])
>>> var.ndim
2
>>> varshape
(2, 6)
You can change the dimension using .reshape function:
>>> var_ = var.reshape(3, 4)
>>> var_.ndim
2
>>> var_.shape
(3, 4)
The shape method requires that a be a Numpy ndarray. But Numpy can also calculate the shape of iterables of pure python objects:
np.shape([[1,2],[1,2]])
a.shape is just a limited version of np.info(). Check this out:
import numpy as np
a = np.array([[1,2],[1,2]])
np.info(a)
Out
class: ndarray
shape: (2, 2)
strides: (8, 4)
itemsize: 4
aligned: True
contiguous: True
fortran: False
data pointer: 0x27509cf0560
byteorder: little
byteswap: False
type: int32
rows = a.shape[0] # 2
cols = a.shape[1] # 2
a.shape #(2,2)
a.size # rows * cols = 4
Execute below code block in python notebook.
import numpy as np
a = np.array([[1,2],[1,2]])
print(a.shape)
print(type(a.shape))
print(a.shape[0])
output
(2, 2)
<class 'tuple'>
2
then you realized that a.shape is a tuple.
so you can get any dimension's size by a.shape[index of dimention]