I have two 3d np.arrays containing numbers.
both np.arrays can have different shapes (different dimensions).
my objective would be to generate a 3d np.array:
which have a shape which contain both other shapes (ie (1,1,3) & (1,2,1) => (1,2,3))
where each element is the sum of the element of the parent 3d array which have the same coordinates (assuming 0 when the coordinates did not exist)
to summarize, I would like to obtain the following:
a=np.array([[[0, 0, 0, 1]]])
b= np.array([[[0],
[1]]])
addition(a, b)
>>> array([[[0, 0, 0, 1],
[1, 0, 0, 0]]])
Thanks in advance for your help
EDIT: I found better
def addition(a,b):
c = np.zeros(np.max([np.shape(a), np.shape(b)], axis=0), dtype=int)
c[np.where(a!=0)] += a[np.where(a!=0)]
c[np.where(b!=0)] += b[np.where(b!=0)]
return c
OLD:
After multiple research, i haven’t found a good way to do it without
iterate over all of the array :
def addition(a, b):
c = np.zeros(np.max([np.shape(a), np.shape(b)], axis=0), dtype=int)
for index, element in np.ndenumerate(a):
c[index] += element
for index, element in np.ndenumerate(b):
c[index] += element
return c
I’ll continue to search a better way to do it
EDIT 2:
I added a dtype=int, because you seem to want to keep the int version instead of the float.
Have fun
Related
I'm currently doing some work to extract NoData values from a gridded satellite image. The image is presented as a 2D array, where the inner array is every pixel value in a given row from left to right, and the outer array is every row in the image from top to bottom.
Any advice on this?
I have built the following functions:
from more_itertools import locate
def find_indices(liste, item):
indices = locate(liste, lambda x: x == item)
return list(indices)
def find_indices2(liste, item):
indices = locate(liste, lambda x: item in x)
return list(indices)
and I have built two separate arrays of the index positions of:
a) the rows containing a '0' value in them (all of them). This is a 1D array marked as 'f'
b) the pixels with a '0' value within their given row. This is a 2D array, marked as 'g'
Finally, I carried out the following to merge my two arrays.
h = np.dstack((g, f))
Which gives me a 3D array of the form [g, list([f])]. I.e. [[0, list([0, 1, 2, 3, 4, 5...])], [1, list([0, 1, 2, 3, 4, 5...])]].
I want to convert this array into the form [[g, f]]. I.e. [[0, 0], [0,1], [0,2], [0,3], [0,4]...]. This will essentially give me a set of 2D co-ordinates for each NoData pixel which I can then apply to a second satellite pixel to mask it, turn both satellite images to arrays of the same length and run a regression on them.
Assuming I understood correctly what you mean, you could do something like this to convert your data:
data = np.array([[0, list([0, 1, 2, 3])], [1, list([0, 1, 2])]])
for i in range(data.shape[0]):
converted = np.asarray(np.meshgrid(data[i][0],data[i][1])).T.reshape(-1,2)
print(converted)
# and you could vstack here for example
This would give the output:
[[0 0]
[0 1]
[0 2]
[0 3]]
[[1 0]
[1 1]
[1 2]]
This can surely be done faster and more efficiently, but you didn't provide exact information on the data you start with. So I'm just trying to address the conversion part of the question. I think it's a bad idea to store data as list inside a numpy array in the first place, especially if its length varies.
I'm trying to add a border (filled with 0's) around an existing array.
My code:
a = np.random.random((4, 5))
m, n = a.shape
b = np.zeros((m+1, n+1))
b[1:-1,1:-1] = a
a,b
but I got a error:
ValueError: could not broadcast input array from shape (4,5) into shape (3,4)
Why can't I directly assign values to parts of a 2D numpy array? where is the problem？
You can accomplish your task using np.pad function:
b = np.pad(a, 1)
Shorter code than yours.
Read the documentation of this function for any details.
b[:, [0, -1]] = 0
b[[0, -1], :] = 0
I have a function in which I do some operations and want to speed it up with numba. In my code changing the values in an array with advanced indexing is not working. I think they do say that in the numba documents. But what is a workaround for like numpy.put()?
Here a short example what I want to do:
#example array
array([[ 0, 1, 2],
[ 0, 2, -1],
[ 0, 3, -1]])
changeing the values at given indexes with any method working in numba...to get:
changed values at:[0,0], [1,2], [2,1]
#changed example array by given indexes with one given value (10)
array([[ 10, 1, 2],
[ 0, 2, 10],
[ 0, 10, -1]])
Here what I did in python, but not working with numba:
indexList is a Tuple, which works with numpy.take()
This is the working example python code and the values in the array change to 100.
x = np.zeros((151,151))
print(x.ndim)
indexList=np.array([[0,1,3],[0,1,2]])
indexList=tuple(indexList)
def change(xx,filter_list):
xx[filter_list] = 100
return xx
Z = change(x,indexList)
Now using #jit on the function:
#jit
def change(xx,filter_list):
xx[filter_list] = 100
return xx
Z = change(x,indexList)
Compilation is falling back to object mode WITH looplifting enabled because Function "change" failed type inference due to: No implementation of function Function() found for signature: setitem(array(float64, 2d, C), UniTuple(array(int32, 1d, C) x 2), Literalint)
This error comes up. So I need a workaround for this. numpy.put() is not supported by numba.
I would be greatful for any ideas.
Thankyou
If it's not a problem for your to keep the indexList as an array you can use it in conjunction with for loops in the change function to make it compatible with numba:
indexList = np.array([[0,1,3],[0,1,2]]).T
#njit()
def change(xx, filter_list):
for y, x in filter_list:
xx[y, x] = 100
return xx
change(x, indexList)
Note that the indexList has to be transposed in order to have the y, x coordinates along the 1st axis. In other words, it has to have a shape of (n, 2) rather than (2, n) for the n points to be change. Effectively it's now a list of coordinates: [[0, 0],[1, 1],[3, 2]]
#mandulaj posted the way to go. Here a little different way I went before mandulaj gave his answer.
With this function I get a deprecation warning...so best way to go with #mandulaj and dont forget to transpose the indexList.
#jit
def change_arr(arr,idx,val): # change values in array by np index array to value
for i,item in enumerate(idx):
arr[item[0],item[1]]= val
return arr
I need to select only the non-zero 3d portions of a 3d binary array (or alternatively the true values of a boolean array). Currently I am able to do so with a series of 'for' loops that use np.any, but this does work but seems awkward and slow, so currently investigating a more direct way to accomplish the task.
I am rather new to numpy, so the approaches that I have tried include a) using
np.nonzero, which returns indices that I am at a loss to understand what to do with for my purposes, b) boolean array indexing, and c) boolean masks. I can generally understand each of those approaches for simple 2d arrays, but am struggling to understand the differences between the approaches, and cannot get them to return the right values for a 3d array.
Here is my current function that returns a 3D array with nonzero values:
def real_size(arr3):
true_0 = []
true_1 = []
true_2 = []
print(f'The input array shape is: {arr3.shape}')
for zero_ in range (0, arr3.shape[0]):
if arr3[zero_].any()==True:
true_0.append(zero_)
for one_ in range (0, arr3.shape[1]):
if arr3[:,one_,:].any()==True:
true_1.append(one_)
for two_ in range (0, arr3.shape[2]):
if arr3[:,:,two_].any()==True:
true_2.append(two_)
arr4 = arr3[min(true_0):max(true_0) + 1, min(true_1):max(true_1) + 1, min(true_2):max(true_2) + 1]
print(f'The nonzero area is: {arr4.shape}')
return arr4
# Then use it on a small test array:
test_array = np.zeros([2, 3, 4], dtype = int)
test_array[0:2, 0:2, 0:2] = 1
#The function call works and prints out as expected:
non_zero = real_size(test_array)
>> The input array shape is: (2, 3, 4)
>> The nonzero area is: (2, 2, 2)
# So, the array is correct, but likely not the best way to get there:
non_zero
>> array([[[1, 1],
[1, 1]],
[[1, 1],
[1, 1]]])
The code works appropriately, but I am using this on much larger and more complex arrays, and don't think this is an appropriate approach. Any thoughts on a more direct method to make this work would be greatly appreciated. I am also concerned about errors and the results if the input array has for example two separate non-zero 3d areas within the original array.
To clarify the problem, I need to return one or more 3D portions as one or more 3d arrays beginning with an original larger array. The returned arrays should not include extraneous zeros (or false values) in any given exterior plane in three dimensional space. Just getting the indices of the nonzero values (or vice versa) doesn't by itself solve the problem.
Assuming you want to eliminate all rows, columns, etc. that contain only zeros, you could do the following:
nz = (test_array != 0)
non_zero = test_array[nz.any(axis=(1, 2))][:, nz.any(axis=(0, 2))][:, :, nz.any(axis=(0, 1))]
An alternative solution using np.nonzero:
i = [np.unique(_) for _ in np.nonzero(test_array)]
non_zero = test_array[i[0]][:, i[1]][:, :, i[2]]
This can also be generalized to arbitrary dimensions, but requires a bit more work (only showing the first approach here):
def real_size(arr):
nz = (arr != 0)
result = arr
axes = np.arange(arr.ndim)
for axis in range(arr.ndim):
zeros = nz.any(axis=tuple(np.delete(axes, axis)))
result = result[(slice(None),)*axis + (zeros,)]
return result
non_zero = real_size(test_array)
I have numpy data which I am trying to turn into contour plot data. I realize this can be done through matplotlib, but I am trying to do this with just numpy if possible.
So, say I have an array of numbers 1-10, and and I want to divide the array according to contour "levels". I want to turn the input array into an array of boolean arrays, each of those being the size of the input, with a 1/True for any data point in that contour level and 0/False everywhere else.
For example, suppose the input is:
[1.2,2.3,3.4,2.5]
And the levels are [1,2,3,4],
then the return should be:
[[1,0,0,0],[0,1,0,1],[0,0,1,0]]
So here is the start of an example I whipped up:
import numpy as np
a = np.random.rand(3,3)*10
print(a)
b = np.zeros(54).reshape((6,3,3))
levs = np.arange(6)
#This is as far as I've gotten:
bins = np.digitize(a, levs)
print(bins)
I can use np.digitize to find out which level each value in a should belong to, but that's as far as I get. I'm fairly new to numpy and this really has me scratching me head. Any help would be greatly appreciated, thanks.
We could gather the indices off np.digitize output, which would represent the indices along the first n-1 axes, where n is the no. of dims in output to be set in the output as True values. So, we could use indexing after setting up the output array or we could use a outer range comparison to achieve the same upon leverage broadcasting.
Hence, with broadcasting one that covers generic n-dim arrays -
idx = np.digitize(a, levs)-1
out = idx==(np.arange(idx.max()+1)).reshape([-1,]+[1]*idx.ndim)
With indexing-based one re-using idx from previous method, it would be -
# https://stackoverflow.com/a/46103129/ #Divakar
def all_idx(idx, axis):
grid = np.ogrid[tuple(map(slice, idx.shape))]
grid.insert(axis, idx)
return tuple(grid)
out = np.zeros((idx.max()+1,) + idx.shape,dtype=int) #dtype=bool for bool array
out[all_idx(idx,axis=0)] = 1
Sample run -
In [77]: a = np.array([1.2,2.3,3.4,2.5])
In [78]: levs = np.array([1,2,3,4])
In [79]: idx = np.digitize(a, levs)-1
...: out = idx==(np.arange(idx.max()+1)).reshape([-1,]+[1]*idx.ndim)
In [80]: out.astype(int)
Out[80]:
array([[1, 0, 0, 0],
[0, 1, 0, 1],
[0, 0, 1, 0]])