I have an 3D matrix a,like this:
a=np.array([[[1,2],[2,3]],[[3,4],[4,5]]])
[
[[1 2],[2 3]]
[[3 4],[4 5]]
]
a.shape
(2, 2, 2)
Now, I want to add another element, like [[5,6],[6,7]] to this array.
So, new array will be:
[
[[1, 2],[2, 3]]
[[3, 4],[4, 5]]
[[5, 6],[6, 7]]
]
a.shape
(3, 2, 2)
What is the best way to do this?
( I'm working with big datasets so I need the best way)
Use np.vstack to vertically stack after extending the second array to 3D by adding a new axis as its first axis with None/np.newaxis, like so -
np.vstack((a,b[None]))
Sample run -
In [403]: a
Out[403]:
array([[[1, 2],
[2, 3]],
[[3, 4],
[4, 5]]])
In [404]: b
Out[404]:
array([[5, 6],
[6, 7]])
In [405]: np.vstack((a,b[None]))
Out[405]:
array([[[1, 2],
[2, 3]],
[[3, 4],
[4, 5]],
[[5, 6],
[6, 7]]])
You can use np.append to append to matrixes:
a = np.array([[[1,2],[2,3]],[[3,4],[4,5]]])
a = np.append(a, [[[5,6],[6,7]]], axis=0)
Note that I had to add an extra set of brackets around the second part, in order for the dimensions to be correct. Also, you must use an axis or it will all be flattened to a linear array.
Try numpy.append
import numpy as np
a=np.array([[[1,2],[2,3]],[[3,4],[4,5]]])
b=np.array([[3,4],[4,5]])
np.append(a,[b[:,:]],axis=0)
Related
I want to add one numpy array two another so it will look like this:
a = [3, 4]
b = [[6, 5], [2, 1]]
output:
[[3, 4], [[6, 5], [2, 1]]]
It should look like the output above and not like [[3,4],[6,5],[2,1]].
How do I do that with numpy arrays?
Work with pure python lists, and not numpy arrays.
It doesn't make sense to have a numpy array holding two list objects. There's literally no gain in doing that.
If you directly instantiate an array like so:
np.array([[3, 4], [[6, 5], [2, 1]]])
You get
array([[3, 4],
[list([6, 5]), list([2, 1])]], dtype=object)
which is an array with dtype=object. Most of numpy's power is lost in this case. For more information on examples and why, take a look at this thread.
If you work with pure python lists, then you can easily achieve what you want:
>>> a + b
[[3, 4], [[6, 5], [2, 1]]]
Numpy as built-in stack commands that are (in my opinion) slightly easier to use:
>>> a = np.array([3, 4])
>>> b = np.array([[6, 5], [2, 1]])
>>> np.row_stack([b, a])
array([[3, 4],
[6, 5],
[2, 1]])
There's also a column stack.
Ref: https://numpy.org/doc/stable/reference/generated/numpy.ma.row_stack.html
You can't stack arrays of different shapes in one array the way you want (or you have to fill in gaps with NaNs or zeroes), so if you want to iterate over them, consider using list.
a = np.array([3, 4])
b = np.array([[6, 5], [2, 1]])
c = [a, b]
for arr in c:
...
If you still want a numpy array, you can try this:
>>> a = np.array([3, 4])
>>> b = np.array([[6, 5], [2, 1]])
>>> a.resize(b.shape,refcheck=False)
>>> c = np.array([a, b])
array([[[3, 4],
[0, 0]],
[[6, 5],
[2, 1]]])
I have an array of arrays:
a = [ [[1,2], [3,4]], [[5,6], [7,8]] ]
How can I, for each subarray in the above array, perform the following:
reshape(-1,1)
concatenate along axis=1
a is not a numpy array but it can of course be converted to a numpy array. I do not care whether it stays a Python list or is converted to a numpy array - whatever yields the most concise/clean solution.
I am aware that this can be accomplished using a for loop but it seems clumsy and I am interesting in cleaner solutions :)
The result I expect is:
[[1, 5], [2, 6], [3, 7], [4, 8]]
The library einops contains a function rearrange that should do the trick in a tidy manner. It uses the same type of indexing as np.einsum to operate flexible reshaping operations.
import numpy as np
a = [ [[1,2], [3,4]], [[5,6], [7,8]] ]
an = np.array(a)
from einops import rearrange
ar = rearrange(an, "a b c -> (b c) a")
ar
output:
array([[1, 5],
[2, 6],
[3, 7],
[4, 8]])
You are looking to reshape each sub-array to (-1, 1), which means the overall shape of the initial array is (len(a), -1, 1). From there we can np.concatenate on axis=1:
>>> a
array([[[1, 2],
[3, 4]],
[[5, 6],
[7, 8]]])
>>> a.reshape((len(a), -1, 1))
array([[[1],
[2],
[3],
[4]],
[[5],
[6],
[7],
[8]]])
>>> np.concatenate(x.reshape((len(x), -1, 1)), 1)
array([[1, 5],
[2, 6],
[3, 7],
[4, 8]])
Note: len(a) is essentially a.shape[0]
Alternatively you can discard the last axis and work with np.stack instead:
>>> np.stack(x.reshape((len(x), -1)), 1)
array([[1, 5],
[2, 6],
[3, 7],
[4, 8]])
A python list solution:
In [182]: alist = []
...: for xy in zip(*a):
...: alist.extend(list(zip(*xy)))
...:
In [183]: alist
Out[183]: [(1, 5), (2, 6), (3, 7), (4, 8)]
I have two feature arrays, e.g.
a = [1, 2, 3]
b = [4, 5, 6]
Now I want to combine these arrays in the following way:
[[1, 4], [2, 5], [3, 6]]
The location in the array corresponds to a timestep. I tried appending and then reshaping, but then I get:
[[1, 2], [3, 4], [5, 6]]
you can use np.dstack to stack your lists depth-wise:
>>> np.dstack([a, b])
array([[[1, 4],
[2, 5],
[3, 6]]])
As noted by #BramVanroy, this does add an unwanted dimension. Two ways around that are to squeeze the result, or to use column_stack instead:
np.dstack([a, b]).squeeze()
# or
np.column_stack([a, b])
Both of which return:
array([[1, 4],
[2, 5],
[3, 6]])
As an alternative to sacuL's reply, you can also simply do
>>> np.array(list(zip(a, b)))
array([[1, 4],
[2, 5],
[3, 6]])
In fact, this is closer to the expected result in terms of the number of dimensions (two, rather than three in sacuL's answer which you still need to .squeeze() to achieve the correct result).
Suppose I have the following (in python 3.7)
x = np.array([2,4,6])
y = np.array([3,5])
How can I obtain the output
np.array([[2, 2],
[3, 4],
[3, 5]])
Basically making use of the two arrays as the "axis" of my new matrix, and for each entry of the new matrix, take the min(row, col), without using any loops.
The function np.meshgrid will expand both of these input variables into 2d arrays (returning a 2-element list); you can then use np.minimum to obtain element-by-element minima:
np.meshgrid(y,x)
returns:
[
array([[3, 5],
[3, 5],
[3, 5]]),
array([[2, 2],
[4, 4],
[6, 6]])
]
and:
np.minimum(*np.meshgrid(y,x))
returns:
array([[2, 2],
[3, 4],
[3, 5]])
(Using * here to expand the list into two separate arguments to np.minimum.)
I need to slice the same element in 3D numpy array (actually masked array, but works the same). I usually do it with iterations - however current data is so huge and it needs repeating the process on thousands of datasets - it will take weeks (raw estimation). What is the quickest way to slice 3D array without looping through all 2D arrays?
In this simple example I need to slice [1, 0] element in each 2D array which is 3 in all 2D arrays and store them in result array.
NetCDF example (slicing element [500, 400])
import netCDF4
url = "http://eip.ceh.ac.uk/thredds/dodsC/public-chess/PET/aggregation/PETAggregation.ncml"
dataset = netCDF4.Dataset(url)
result = dataset.variables['pet'][:, 500, 400]
myarray SUPERSEDED NOW WITH ABOVE
myarray = np.array([
[[1, 2], [3, 4], [5, 6]],
[[1, 2], [3, 4], [5, 6]],
[[1, 2], [3, 4], [5, 6]],
[[1, 2], [3, 4], [5, 6]],
])
result = []
for i in myarray:
result.append(i[1][0])
result [3, 3, 3, 3]
EDIT
FirefoxMetzger suggested to slice it simply with
result = myarray[:, 1, 0]. However, I'm getting the following error message with this:
RuntimeError: NetCDF: DAP server error
The minimal numpy example you provided can be efficiently sliced using standard slicing mechanisms:
myarray = np.array([
[[1, 2], [3, 4], [5, 6]],
[[1, 2], [3, 4], [5, 6]],
[[1, 2], [3, 4], [5, 6]],
[[1, 2], [3, 4], [5, 6]],
])
result = myarray[:, 1, 0]
The NetCFD seems to come from the resulting slice being too large to be returned from the server, causing a crash. As per your comment, the solution here is to query the server in chunks and aggregate the results locally.