I am trying to create 3D array in python using Numpy and by multiplying 2D array in to 3rd dimension. I am quite new in Numpy multidimensional arrays and basically I am missing something important here.
In this example I am trying to make 10x10x20 3D array using base 2D array(10x10) by copying it 20 times.
My starting 2D array:
a = zeros(10,10)
for i in range(0,9):
a[i+1, i] = 1
What I tried to create 3D array:
b = zeros(20)
for i in range(0,19):
b[i]=a
This approach is probably stupid. So what is correct way to approach construction of 3D arrays from base 2D arrays?
Cheers.
Edit
Well I was doing things wrongly probably because of my R background.
Here is how I did it finally
b = zeros(20*10*10)
b = b.reshape((20,10,10))
for i in b:
for m in range(0, 9):
i[m+1, m] = 1
Are there any other ways to do the same?
There are many ways how to construct multidimensional arrays.
If you want to construct a 3D array from given 2D arrays you can do something like
import numpy
# just some 2D arrays with shape (10,20)
a1 = numpy.ones((10,20))
a2 = 2* numpy.ones((10,20))
a3 = 3* numpy.ones((10,20))
# creating 3D array with shape (3,10,20)
b = numpy.array((a1,a2,a3))
Depending on the situation there are other ways which are faster. However, as long as you use built-in constructors instead of loops you are on the fast side.
For your concrete example in Edit I would use numpy.tri
c = numpy.zeros((20,10,10))
c[:] = numpy.tri(10,10,-1) - numpy.tri(10,10,-2)
Came across similar problem...
I needed to modify 2D array into 3D array like so:
(y, x) -> (y, x, 3).
Here is couple solutions for this problem.
Solution 1
Using python tool set
array_3d = numpy.zeros(list(array_2d.shape) + [3], 'f')
for z in range(3):
array_3d[:, :, z] = array_2d.copy()
Solution 2
Using numpy tool set
array_3d = numpy.stack([array_2d.copy(), ]*3, axis=2)
That is what I came up with. If someone knows numpy to give a better solution I would love to see it! This works but I suspect there is a better way performance-wise.
Related
I use numpy to do image processing, I wanted to switch the image to black and white and for that I did the calculation in each cell to see the luminosity, but if i want to show it i have to transform a 2d array into 2d array with 3 times the same value
for exemple i have this:
a = np.array([[255,0][0,255]])
#into
b = np.array([[[255,255,255],[0,0,0]],[[0,0,0],[255,255,255]]])
I've been searching for a while but i don't find anything to help
PS: sorry if i have made some mistake with my English.
You'll want to us an explicit broadcast: https://numpy.org/doc/stable/reference/generated/numpy.broadcast_to.html#numpy.broadcast_to
b = np.broadcast_to(a[..., np.newaxis], (2, 2, 3))
Usually you don't need to do it explicitly, maybe try and see if just a[..., np.newaxis] and the standard broadcasting rules are enough.
Another way to do it
np.einsum('ij,k->ijk', a, [1,1,1])
It's a way to create a 3 dimensional array (hence the ijk), from a 2d array (ij) and a 1d array (k). Whose result is for all i,j,k being indices of a and of [1,1,1], the 3d matrix of a[i,j]×[1,1,1][k].
I want to create a 2D numpy array of size (N_r * N_z).
Across columns, the elements for 1 specific column (say j) shall be created based on the value r_thresh[j].
So 1 column (say j) out of the total of N_z columns in the numpy 2D array is created as:
(np.arange(N_r) + 0.5) * r_thresh[j] # this gives an array of size (1, N_r)
Of course, the column j + 1 shall be created as:
(np.arange(N_r) + 0.5) * r_thresh[j+1] # this gives an array of size (1, N_r)
r_thresh is a numpy array of size (1, N_z), already populated with values before I want to create the 2D array.
I want to ask you how do I go further and use this ''rule'' of creating each element of the numpy 2D array and actually create the whole array, in the most efficient way possible (speed-wise).
I initially wrote all the code using 2 nested for loops and plain python lists and the code worked, but took forever to run.
More experienced programmers told me to avoid for loops and use numpy because it's the best.
I now understand how to create 1D arrays using numpy np.arange() instruction, but I lack the knowledge on how to extrapolate this to 2 Dimensions.
Thanks!
The easiest way is to use einsum. In the case of r_thresh with the shape of (N_z,), you can use this code:
res = np.einsum("i,j->ij", np.arange(N_r) + 0.5, r_thresh)
Also, you can reshape np.arange(N_r) + 0.5 to the shape (N_r,1) and r_thresh to the shape (1,N_z). Thus, you can use the dot product (for Python version > 3.5):
res = (np.arange(N_r) + 0.5).reshape(N_r,1) # r_thresh.reshape(1,N_z)
or following to the comment of hpaulj:
res = (np.arange(N_r) + 0.5)[:,None] # r_thresh[None,:]
EDIT1
The comment of hpaulj is also very helpful (I pasted this into my answer to see better):
res = (np.arange(N_r) + 0.5)[:,None] * r_thresh
res = np.outer(np.arange(N_r) + 0.5, r_thresh)
IN ADDITION
You can also use tensordot:
res = np.tensordot((np.arange(N_r) + 0.5)[:,None], r_thresh[:,None], axes=[[-1],[-1]])
I have two numpy arrays, with just the 3-dimensional coordinates of two molecules.
I need to implement the following equation, and I'm having problems in the subtraction of each coordinate of one of the arrays by the second, and then square it.
I have tried the following, but since I'm still learning I feel that I am making some major mistake. The simple code I use is:
a = [math.sqrt(1/3*((i[:,0]-j[:,0])**2) + ((i[:,1] - j[:,1])**2) + ((i[:,2]-j[:,2])**2) for i, j in zip(coordenates_2, coordenates_1))]
It's numpy you can easily do it using the following example:
import numpy as np
x1 = np.random.randn(3,3,3)
x2 = np.random.randn(3,3,3)
res = np.sqrt(np.mean(np.power(x1-x2,2)))
I am trying to sample with replacement a base 2D numpy array with shape of (4,2) by rows, say 10 times. The final output should be a 3D numpy array.
Have tried the code below, it works. But is there a way to do it without the for loop?
base=np.array([[20,30],[50,60],[70,80],[10,30]])
print(np.shape(base))
nsample=10
tmp=np.zeros((np.shape(base)[0],np.shape(base)[1],10))
for i in range(nsample):
id_pick = np.random.choice(np.shape(base)[0], size=(np.shape(base)[0]))
print(id_pick)
boot1=base[id_pick,:]
tmp[:,:,i]=boot1
print(tmp)
Here's one vectorized approach -
m,n = base.shape
idx = np.random.randint(0,m,(m,nsample))
out = base[idx].swapaxes(1,2)
Basic idea is that we generate all the possible indices with np.random.randint as idx. That would an array of shape (m,nsample). We use this array to index into the input array along the first axis. Thus, it selects random rows off base. To get the final output with a shape (m,n,nsample), we need to swap last two axes.
You can use the stack function from numpy. Your code would then look like:
base=np.array([[20,30],[50,60],[70,80],[10,30]])
print(np.shape(base))
nsample=10
tmp = []
for i in range(nsample):
id_pick = np.random.choice(np.shape(base)[0], size=(np.shape(base)[0]))
print(id_pick)
boot1=base[id_pick,:]
tmp.append(boot1)
tmp = np.stack(tmp, axis=-1)
print(tmp)
Based on #Divakar 's answer, if you already know the shape of this 2D-array, you can treat it as an (8,) 1D array while bootstrapping, and then reshape it:
m, n = base.shape
flatbase = np.reshape(base, (m*n,))
idxs = np.random.choice(range(8), (numReps, m*n))
bootflats = flatbase[idx]
boots = np.reshape(flatbase, (numReps, m, n))
I'm currently trying to find an easy way to do the following operation to an N dimensional array in Python. For simplicity let's start with a 1 dimensional array of size 4.
X = np.array([1,2,3,4])
What I want to do is create a new array, call it Y, such that:
Y = np.array([1,2,3,4],[2,3,4,1],[3,4,1,2],[4,1,2,3])
So what I'm trying to do is create an array Y such that:
Y[:,i] = np.roll(X[:],-i, axis = 0)
I know how to do this using for loops, but I'm looking for a faster method of doing so. The actual array I'm trying to do this to is a 3 dimensional array, call it X. What I'm looking for is a way to find an array Y, such that:
Y[:,:,:,i,j,k] = np.roll(X[:,:,:],(-i,-j,-k),axis = (0,1,2))
I can do this using the itertools.product class using for loops, but this is quite slow. If anyone has a better way of doing this, please let me know. I also have CUPY installed with a GTX-970, so if there's a way of using CUDA to do this faster please let me know. If anyone wants some more context please let me know.
Here is my original code for computing the position space two point correlation function. The array x0 is an n by n by n real valued array representing a real scalar field. The function iterate(j,s) runs j iterations. Each iteration consists of generating a random float between -s and s for each lattice site. It then computes the change in the action dS and accepts the change with a probability of min(1,exp^(-dS))
def momentum(k,j,s):
global Gxa
Gx = numpy.zeros((n,n,t))
for i1 in range(0,k):
iterate(j,s)
for i2,i3,i4 in itertools.product(range(0,n),range(0,n),range(0,n)):
x1 = numpy.roll(numpy.roll(numpy.roll(x0, -i2, axis = 0),-i3, axis = 1),-i4,axis = 2)
x2 = numpy.mean(numpy.multiply(x0,x1))
Gx[i2,i3,i4] = x2
Gxa = Gxa + Gx
Gxa = Gxa/k
Approach #1
We can extend this idea to our 3D array case here. So, simply concatenate with sliced versions along the three dims and then use np.lib.stride_tricks.as_strided based scikit-image's view_as_windows to efficiently get the final output as the strided-view of the concatenated version, like so -
from skimage.util.shape import view_as_windows
X1 = np.concatenate((X,X[:,:,:-1]),axis=2)
X2 = np.concatenate((X1,X1[:,:-1,:]),axis=1)
X3 = np.concatenate((X2,X2[:-1,:,:]),axis=0)
out = view_as_windows(X3,X.shape)
Approach #2
For really large arrays, we might want to initialize the output array and then re-use X3 from earlier approach to assign with slicing it. This slicing process would be faster than the original-rolling. The implementation would be -
m,n,r = X.shape
Yout = np.empty((m,n,r,m,n,r),dtype=X.dtype)
for i in range(m):
for j in range(n):
for k in range(r):
Yout[:,:,:,i,j,k] = X3[i:i+m,j:j+n,k:k+r]