I have a large M x N 2D matrix of positive coefficients (8bit unsigned) with M,N~10^3.
I want to optimize the parameters (M*N~10^6) such that my error-function (where I put the matrix in) is minimized. Boundary conditions: Neighboring parameters vary slowly/smoothly.
I have already tried to downscale the matrix in order to reduce the number of parameters and then flatten the matrix to feed it into Scipy's Minimize function. This causes memory errors quickly.
Is there a smart approch to optimize this large coefficient matrix in time ranges less than infinity without applying a low-parametric model?
The Fast Fourier Transform (fft; documentation) transforms 'a' into its fourier, spectral equivalent:
numpy.fft.fft(a, n=None, axis=-1, norm=None)
The parameter, n represents—so far as I understand it—how many samples are in the output, where the output is either cropped if n is smaller than the number of samples in a, or padded with zeros if n is larger.
What does axis do? What does it mean exactly? I haven't been able to find any clear examples of its use.
np.fft.fft computes the one-dimensional discrete Fourier transform. If you give a one dimensional input (a vector), it will just compute the transform for that input. However, if your input has more than one dimension, like a 2D matrix, or higher, NumPy assumes you are giving many vectors and you want to compute the transform of each of them. The axis parameter indicates the dimension corresponding to those vectors, and by default it is the last one (-1). So, for example, for a 2D matrix m, if axis=0 then each column m[:, 0], m[:, 1], etc. would be the vectors for which the transform is computed, while passing axis=1 (equivalent to the default axis=-1), each row m[0, :], m[1, :], etc. would be considered a vector for the transform. If you want to compute the transform of all values in the input, regardless of the dimensions, you would have to flatten the input, for example with np.ravel.
Btw, this is a very common convention in NumPy (and many other algebra packages), where a one-dimensional operation can work on multidimensional inputs by receiving an axis parameter that indicates the dimension over which the operation is performed.
numpy.fft.fft() returnes a one-dimensional fourier-transform of your array. That means if you have an array of shape (N,M) it will not give you a two-dimensional fft (np.fft.fft2() does) but return the fft along the last axis. If you like to have the fft calculated rather along the columns than the rows you should pass axis=0.
I have a basis set of square matrices and a data set that I need to find the coefficients for given that my data is a linear sum of the basis set.
def basis(a,b,c):
return a*gam1+b*gam2+c*kapp+allsky
So data = basis and I need the best fit (least square) values of the coefficients a,b and c. The basis matrices and the data matrices are all square matrices of 89x89. I have tried using np.linalg.lstsq however since my A matrix would need to be a matrix of the 4 basis matrices the array dimension becomes 4 and throws an error stating the array dimension must be 2. Any help is appreciated.
I have a 2D array of values and I'm trying to analyze spatial correlations. To calculate a 2D autocorrelation like Moran's I in python, pysal provides an implementation.
1) How do I transform my 2D data into a 1D array expected by pysal?
2) How do I construct a weight array w that is based on distance (what does the input array of points mean in the Kernel distance function?)?
1) The weights array should be flattened in the same way as you flatten the data array. The order doesn't matter, as long as the indices agree.
2) The input array can be spatial coordinates (e.g. x and y, or lat and long). By far the easiest are the indices of your original matrix (e.g. 1 to n times 1 to m).
In the end, your data will be a list with 3 elements: x, y and value. Your weights will be a list with 5 elements: x_from, y_from, x_to, y_to and weight.
I have a set of numpy.arrays of NXM (two dimensions: Range and Azimuth).
I need to form a stack of three dimensions and extract a single dimension vector to compute a covariance matrix (the red vectors in the picture).
How i do this efficiently and easy in Python?
You can make a 3D numpy array pretty easily and then just use the indexing to pull out the bits that you're interested in:
stackOfImages = np.array((image1, image2)) #iterate over these if many more
redData = stackOfImages[:, N-1, M-1]