I am currently trying to optimize a function of the form:
where sigma and x are the optimization variables. We shall say that x is a vector (of the form [x_1, x_2]) that holds the coefficients to X_2 = x_1^2 * A + x_2^2 * B where A and B are positive matrices.
I have attempted to use scipy.optimize but it isn't working. I'm certain that it has to do with the optimization variables being a scalar/matrix. I am attaching my code below:
from scipy.optimize import minimize
import numpy as np
def objective(x):
x_array = x[0]
sigma = x[1]
fun = np.trace((np.kron(np.diag(np.square(x_array)), rho.T)) # sigma)
return fun
x0 = np.array([np.array([1 ,1]), np.eye(4)])
res = minimize(objective, x0, method = 'Nelder-Mead')
I am getting an error that says
ValueError: setting an array element with a sequence.
How can I go about solving this optimization problem in Python?
In scipy.optimize the x vector should be 1-dimensional vector of length NUMVAR. You pass on something more complicated. You should do something like:
from scipy.optimize import minimize
import numpy as np
n1 = 2 # number of variables in user x array
n2 = 4 # number of variables in sigma
n = n1 + n2 # number of variables in scipy.optimize x array
def objective(x):
#x_array = x[0]
#sigma = x[1]
x_array = x[0:n1]
sigma = np.diag(x[n1:n])
fun = np.trace((np.kron(np.diag(np.square(x_array)), rho.T)) # sigma)
return fun
# x0 = np.array([np.array([1 ,1]), np.eye(4)])
# this is a very strange beast. How does one come up with this?
# even numpy thinks this is just bonkers:
# VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences
# (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated.
# If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
# let's bring some sanity back and just use a vector of length n=n1+n2
x0 = np.ones((n,))
res = minimize(objective, x0, method = 'Nelder-Mead')
This does not run as we don't have a proper running minimal example. Next time, please have a look at: https://stackoverflow.com/help/minimal-reproducible-example.
Note that I assumed we have 2 variables for x and 4 for sigma and that sigma is a diagonal matrix. If sigma is a full matrix it will have 16 variables. Not clear what the user intended. This is a good example where it is always a good idea to state the problem in clear math first. I don't see any A and B matrices in the code, and in the mathematical model, so the description seems to be out of sync. Of course, we are also missing any constraints. Again, the description is not the same as the math which is not the same as the code.
I am trying to construct a stack of block diagonal matrix in the form of nXMXM in numpy/scipy from a given stacks of matrices (nXmXm), where M=k*m with k the number of stacks of matrices. At the moment, I'm using the scipy.linalg.block_diag function in a for loop to perform this task:
import numpy as np
import scipy.linalg as linalg
a = np.ones((5,2,2))
b = np.ones((5,2,2))
c = np.ones((5,2,2))
result = np.zeros((5,6,6))
for k in range(0,5):
result[k,:,:] = linalg.block_diag(a[k,:,:],b[k,:,:],c[k,:,:])
However, since n is in my case getting quite large, I'm looking for a more efficient way than a for loop. I found 3D numpy array into block diagonal matrix but this does not really solve my problem. Anything I could imagine is transforming each stack of matrices into block diagonals
import numpy as np
import scipy.linalg as linalg
a = np.ones((5,2,2))
b = np.ones((5,2,2))
c = np.ones((5,2,2))
a = linalg.block_diag(*a)
b = linalg.block_diag(*b)
c = linalg.block_diag(*c)
and constructing the resulting matrix from it by reshaping
result = linalg.block_diag(a,b,c)
result = result.reshape((5,6,6))
which does not reshape. I don't even know, if this approach would be more efficient, so I'm asking if I'm on the right track or if somebody knows a better way of constructing this block diagonal 3D matrix or if I have to stick with the for loop solution.
Edit:
Since I'm new to this platform, I don't know where to leave this (Edit or Answer?), but I want to share my final solution: The highlightet solution from panadestein worked very nice and easy, but I'm now using higher dimensional arrays, where my matrices reside in the last two dimensions. Additionally my matrices are no longer of the same dimension (mostly a mixture of 1x1, 2x2, 3x3), so I adopted V. Ayrat's solution with minor changes:
def nd_block_diag(arrs):
shapes = np.array([i.shape for i in arrs])
out = np.zeros(np.append(np.amax(shapes[:,:-2],axis=0), [shapes[:,-2].sum(), shapes[:,-1].sum()]))
r, c = 0, 0
for i, (rr, cc) in enumerate(shapes[:,-2:]):
out[..., r:r + rr, c:c + cc] = arrs[i]
r += rr
c += cc
return out
which works also with array broadcasting, if the input arrays are shaped properly (i.e. the dimensions, which are to be broadcasted are not added automatically). Thanks to pandestein and V. Ayrat for your kind and fast help, I've learned a lot about the possibilites of list comprehensions and array indexing/slicing!
block_diag also just iterate through shapes. Almost all time spend in copying data so you can do it whatever way your want for example with little change of source code of block_diag
arrs = a, b, c
shapes = np.array([i.shape for i in arrs])
out = np.zeros([shapes[0, 0], shapes[:, 1].sum(), shapes[:, 2].sum()])
r, c = 0, 0
for i, (_, rr, cc) in enumerate(shapes):
out[:, r:r + rr, c:c + cc] = arrs[i]
r += rr
c += cc
print(np.allclose(result, out))
# True
I don't think that you can escape all possible loops to solve your problem. One way that I find convenient and perhaps more efficient than your for loop is to use a list comprehension:
import numpy as np
from scipy.linalg import block_diag
# Define input matrices
a = np.ones((5, 2, 2))
b = np.ones((5, 2, 2))
c = np.ones((5, 2, 2))
# Generate block diagonal matrices
mats = np.array([a, b, c]).reshape(5, 3, 2, 2)
result = [block_diag(*bmats) for bmats in mats]
Maybe this can give you some ideas to improve your implementation.
i want to solve this linear equation in python
import numpy as np
x2=264
x1=266
x3=294
y2=270
y1=240
y3=227
fract=(x2-x1)*(y3-y1)-(y2-y1)*(x3-x1)
A = np.matrix([[fract-(y3-y1)*(x3-x1)+(y2-y1)*(x2-x1),((x3-x1)**2)-(x2-x1)**2],[((y2-y1)**2)-(y3-y1)**2,fract+(y3-y1)*(x3-x1)-(y2-y1)*(x2-x1)]])
B = np.matrix([[(fract+(y3-y1)*(x3-x1)-(y2-y1)*(x2-x1))], [y1*fract+(y2-y1)*(x1*y2-y1*x2)+(y3-y1)*(x3*y1-y3*x1)]])
A_inverse = np.linalg.inv(A)
X = A_inverse * B
print (X)
LinAlgError: Singular matrix
This is explained simply by printing A:
[[ -510 780]
[ 731 -1118]]
Both cofactors are 570180, so the determinant is 0.
As the error message tells you, the matrix is singular, which means there is no unique solution: either none or infinite, depending on the constants applied.
In the following function, if I use
np.linalg.inv when Nx, Nt get large the function seems to take forever. In my mind I know I should instead use sparse matrices, which are in scipy (which I've never used before), but I'm getting really stuck how to convert M to a sparse matrix, find its inverse, and then convert it back to a numpy array for the for loop.
If anyone could help I'd be really grateful! Thanks!
def BTCS(phiOld, c, Nx, Nt):
#Initiate phi for the for loop
phi = phiOld.copy()
#Crate the matrix M for the BTCS scheme
M = np.zeros((Nx, Nx))
for i in range(Nx):
M[i,(i-1)%Nx] = -c/2
M[i,i] = 1
M[i,(i+1)%Nx] = c/2
#Take the inverse of M so as to have phi(n+1) = M^(-1) * phi(n)
M_inv = np.linalg.inv(M)
#Loop over all time steps
for it in range(Nt):
#Loop over space (excluding end points)
for ix in range(1,Nx-1):
phi[ix] = M_inv.dot(phiOld)[ix]
#Compute boundary values using periodic boundary conditions
phi[0] = M_inv.dot(phiOld)[0]
phi[Nx-1] = phi[0]
#Update old time value
phiOld = phi.copy()
return phi
Given a large sparse matrix A which are banded or tridiagonals (however it is called) and a vector f, I would like to solve for Z, where AZ = f.
There are 6 diagonals, not clearly shown here.
A has more M rows than N columns (just by 1, M ~= N), hence it is over-determined. Here is the source Matlab code, and I would like to convert it to its Scipy equivalent.
Matlab
A = A(:,2:end); #less one column
f = f(:);
Z = A\f;
Z = [0;-Z];
Z = reshape(Z,H,W);
Z = Z - min(Z(:));
My attempt on Scipy gives me this, but solving Z with scipy.sparse.linalg lsqr & lsmr is a lot slower than Matlab \ as well as not giving a good enough solution. A is created as a csr_matrix.
Python
A = A[:,1:]
f = f.flatten(1)
Z = la.lsqr(A, f, atol=1e-6, btol=1e-6)
#Z = la.lsmr(A, f) # the other method i used
Z = Z[0]
Z = np.append([0], np.negative(Z))
Z = np.reshape(Z, (height, width), order='F').copy()
Z = Z - Z.flatten(1).min()
Could anyone recommend a better alternative to solve for Z, that is as effective and fast as Matlab \ ?
This looks like a good candidate for solve_banded.
Unfortunately, the interface for providing the banded matrix is a little complex. You could start by converting your sparse matrix to DIA format, and work from there.