The LU decomposition function provided by scipy returns a permutation matrix P
P,L,U = scipy.linalg.lu(A)
Where A is a rectangular matrix. However the size of my problem do not allow to store P (even temporary) due to its size, I really need a function that computes a permutation vector (like [L,U,P] = lu(A,'vector') in Matlab). I found a lapack function
LU,p,info = scipy.linalg.lapack.dgetrf(A)
which seems to return a vector p but I learned this latter is not an actual permutation vector since it contains twice the same value (https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/780655). I am thus looking for another function (which may be from another library) to perform this LU decomposition with pivoting. Since the computation time is also very important, I don't think implementing the decomposition myself will be efficient.
Yes, it's a pivot vector, which is a standard LAPACK return. So you'll need to convert it into whatever other form you want yourself (and this is much easier then reimplementing the factorization).
Related
I'm currently working with a least-square algorithm on Python, regarding some geodetic calculations.
I chose Python (which is not the fastest) and it works pretty well. However, in my code, I have inversions of large sparse symmetric (non-positive definite, so can't use Cholesky) matrix to execute (image below). I currenty use np.linalg.inv() which is using the LU decomposition method.
I pretty sure there would be some optimization to do in terms of rapidity.
I thought about Cuthill-McKee algotihm to rearange the matrix and take its inverse. Do you have any ideas or advice ?
Thank you very much for your answers !
Good news is that if you're using any of the popular python libraries for linear algebra (namely, numpy), the speed of python really doesn't matter for the math – it's all done natively inside the library.
For example, when you write matrix_prod = matrix_a # matrix_b, that's not triggering a bunch of Python loops to multiply the two matrices, but using numpy's internal implementation (which I think uses the FORTRAN LAPACK library).
The scipy.sparse.linalg module has your back covered when it comes to solving sparsely stored matrices specifying sparse systems of equations. (which is what you do with the inverse of a matrix). If you want to use sparse matrices, that's your way to go – notice that there's matrices that are sparse in mathematical terms (i.e., most entries are 0), and matrices which are stored as sparse matrix, which means you avoid storing millions of zeros. Numpy itself doesn't have sparsely stored matrices, but scipy does.
If your matrix is densely stored, but mathematically sparse, i.e. you're using standard numpy ndarrays to store it, then you won't get any more rapid by implementing anything in Python. The theoretical complexity gains will be outweighed by the practical slowness of Python compared to highly optimized inversion.
Inverting a sparse matrix usually loses the sparsity. Also, you never invert a matrix if you can avoid it at all! For a sparse matrix, solving the linear equation system Ax = b, with A your matrix and b a known vector, for x, is so much faster done forward than computing A⁻¹! So,
I'm currently working with a least-square algorithm on Python, regarding some geodetic calculations.
since LS says you don't need the inverse matrix, simply don't calculate it, ever. The point of LS is finding a solution that's as close as it gets, even if your matrix isn't invertible. Which can very well be the case for sparse matrices!
I'm trying to to find a way to perform an incomplete LU factorization of a symbolic matrix in SymPy and cannot find anything helpful on my own. It's an option for solvers to use ilu as a preconditioner, but it seems there's no way to have it on its own.
Am I missing it? Is it even possible/feasible for symbolic matrices of size 20x20 and larger?
The reason I need this is because I need to approximate O(1) terms in the inverse of that symbolic matrix. I had luck with ilu and non-symbolic matrices, so I thought this may be the way. If this is relevant, the symbols are all binary variables and linear in the terms.
Update 1:
I tried to use the LU solver, but the number of variables in the matrix is much lower than the matrix dimension, so it's no option (unless there is an efficient way to compute just the very first component of the solution vector?). I also tried full LU decomposition with the additional simplification function
def simpfunc(E):
E = E.replace(lambda e: e.is_Pow, lambda e: e.args[0])
return E
which I do hope is correctly formulated this way, since there seems to be no example in the documentation. The idea came from the answer to a previous question here. I could additionally provide an iszerofunc because terms with more than n factors would be zero automatically, but I don't know how to check the degree of terms (example: 0.5x_0x_1x_2x_4 would be zero, while 0.8x_0x_2x_4 would not).
I'm attempting to write a simple implementation of the Newton-Raphson method, in Python. I've already done so using the SymPy library, however what I'm working on now will (ultimately) end up running in an environment where only Numpy is available.
For those unfamiliar with the algorithm, it works (in my case) as follows:
I have some a system of symbolic equations, which I "stack" to form a matrix F. The unknowns are X,Y,Z,T (which I wish to determine). Some additional values are initially unknown, until passed to my solver, which substitutes these known values for variables in the symbolic expressions.
Now, the Jacobian matrix (J) of F is computed. This, too, is a matrix of symbolic expressions.
Now, I iterate in some range (max_iter). With each iteration, I form a matrix A by substituing for the unknowns X,Y,Z,T in F current estimates (starting with some initial values). Similarly, I form a matrix b by substituting for X,Y,Z,T current estimates.
I then obtain new estimates by solving the matrix equation Ax = b for x. This vector x holds dT, dX, dY, dZ. I then add these to current estimates for T,X,Y,Z, and iterate again.
Thus far, I've found my largest issue to be computing the Jacobian matrix. I need only to do this once, however it will be different depending upon the coefficients fed to the solver (not unknowns, but only known once fed to the solver, so I can't simply hard-code the Jacobian).
While I'm not terribly familiar with Numpy, I know that it offers numpy.gradient. I'm not sure, however, that this is the same as SymPy's .jacobian.
How can the Jacobian matrix be found, either in "pure" Python, or with Numpy?
EDIT:
Should it be useful to you, more information on the problem can be found [here]. 1. It can be formulated a few different ways, however (as of now) I'm writing it as 4 equations of the form:
\sqrt{(X-x_i)^2+(Y-y_i)^2+(Z-z_i)^2 }= c * (t_i-T)
Where X,Y,Z and T are unknown.
This describes the solution to a localization problem, where we know (a) the location of n >= 4 observers in a 3-dimensional space, (b) the time at which each observer "saw" some signal, and (c) the velocity of the signal. The goal is to determine the coordinates of the signal source X,Y,Z (and, as a side effect, the time of emission, T).
Notice that I've tried (many) other approaches to solving this problem, and all leads point toward a combination of Newton-Raphson with regression.
Matlab and Julia have the backslash operator that solves linear systems. I don't really know what Matlab does, but Julia does not compute the inverse, but it computes the effect the inverse has on a given vector, which is computationally easier.
I have a numpy sparse matrix and I want to apply its pseudo-inverse to a vector. Does Python have to compute the pseudo-inverse first or is there a backslash-like operator I can use?
Edit: In a sense I want to solve a linear system Ax=b. However the matrix A does not have full rank and the vector b is not in A's range. So the system does not have a solution. So in practice I want to get the vector X that minimises the norm of Ax-b. This is exactly what the pseudo-inverse matrix does. My question is whether I there is a function that will give me that without having to compute the pseudo-inverse first.
Assume that I have a square matrix M. Assume that I would like to invert the matrix M.
I am trying to use the the fractions mpq class within gmpy2 as members of my matrix M. If you are not familiar with these fractions, they are functionally similar to python's built-in package fractions. The only problem is, there are no packages that will invert my matrix unless I take them out of fraction form. I require the numbers and the answers in fraction form. So I will have to write my own function to invert M.
There are known algorithms that I could program, such as gaussian elimination. However, performance is an issue, so my question is as follows:
Is there a computationally fast algorithm that I could use to calculate the inverse of a matrix M?
Is there anything else you know about these matrices? For example, for symmetric positive definite matrices, Cholesky decomposition allows you to invert faster than the standard Gauss-Jordan method you mentioned.
For general matrix inversions, the Strassen algorithm will give you a faster result than Gauss-Jordan but slower than Cholesky.
It seems like you want exact results, but if you're fine with approximate inversions, then there are algorithms which approximate the inverse much faster than the previously mentioned algorithms.
However, you might want to ask yourself if you need the entire matrix inverse for your specific application. Depending on what you are doing it might be faster to use another matrix property. In my experience computing the matrix inverse is an unnecessary step.
I hope that helps!