Penalty function method - python

I am trying to implement penalty function method for minimizing function. I need to find the minimum of Rosenbrok's function.
I am using this penalty function:
First of all, I have found the minimum using scipy.optimize.minimize:
from scipy.optimize import minimize, rosen
rz = lambda x: (1-x[0])**2 + 100*(x[1] - x[0]**2)**2;
h_1 = lambda x: (x[0] - 2 * x[1] + 2);
h_2 = lambda x: (-x[0] - 2 * x[1] + 6);
h_3 = lambda x: (-x[0] + 2 * x[1] + 2);
x0 = [2.3, 5];
cons = ({'type': 'ineq', 'fun': h_1},
{'type': 'ineq', 'fun': h_2},
{'type': 'ineq', 'fun': h_3})
minimize(rz, x0, constraints=cons)
The answer is x: array([ 0.99971613, 0.99942073])
Then I am trying to find the minimum using my implementation of penalty method:
x_c = [2.3, 3];
i = 1;
while i < 1000:
curr_func = lambda x: rz(x) + i*(h_1(x)**2 + h_2(x)**2 + h_3(x)**2)
x_c = minimize(curr_func, x_c).x;
i *= 1.2;
print(answer.x);
Which gives me [ 2.27402022 1.4157964 ] (if I increase the number of iterations, final values are even greater).
Where is the mistake in my implementation?
Thanks.
P.S. Function curr_func is specific for my constraints, of course, when they are all 'inequals' type.

The problem you have is that the h_i in your formula are for equality constraints, whereas the problem you are solving is for inequality constraints, which correspond to the g_i in your formula. Hence, your penalty function should be using terms like min(0, h_1(x))**2 instead of h_1(x)**2. To see why this is the case, just think about what happens if i = 1000 and x is the desired solution (1, 1). Then, the penalty will include a term i * h_1(x)**2 = 1000, which is huge.
Note that I used min instead of max because it seems like the inequality you want to enforce is h_1(x) >= 0. That means as long as h_1(x) >= 0, the penalty should be zero, but as soon as h_1(x) goes negative, you start penalizing. If it's actually h_1(x) <= 0 you want, then you use max (then you'll have to switch h_1 with -h_1 when you use scipy.optimize.minimize).
BTW, since i is usually an index variable, it's probably better to name the penalty weight something else, like a.

Related

Problems minimizing a function with 2 constraints - Python

I'm writing a program to minimize a function of several parameters subjected to constraints and bounds. Just in case you want to run the program, the function is given by:
def Fnret(mins):
Bj, Lj, a, b = mins.reshape(4,N)
S1 = 0; S2 = 0
Binf = np.zeros(N); Linf = np.zeros(N);
for i in range(N):
sbi=(Bi/2); sli=(Li/2)
for j in range(i+1):
sbi -= Bj[j]
sli -= Lj[j]
Binf[i]=sbi
Linf[i]=sli
for i in range(N):
S1 += (C*(1-np.sin(a[i]))+T*np.sin(a[i])) * ((2*Bj[i]*Binf[i]+Bj[i]**2)/(np.tan(b[i])*np.cos(a[i]))) +\
(C*(1-np.sin(b[i]))+T*np.sin(b[i])) * ((2*Bj[i]*Linf[i]+Lj[i]*Bj[i])/(np.sin(b[i])))
S2 += (gamma*Bj[0]/(6*np.tan(b[0])))*((Bi/2)*(Li/2) + 4*(Binf[0]+Bj[0])*(Linf[0]+Lj[0]) + Binf[0]*Linf[0])
j=1
while j<(N):
S2 += (gamma*Bj[j]/(6*np.tan(b[j])))*(Binf[j-1]*Linf[j-1] + 4*(Binf[j]+Bj[j])*(Linf[j]+Lj[j]) + Binf[j]*Linf[j])
j += 1
F = 2*(S1+S2)
return F
where Bj,Lj,a, and b are the minimization results given by N-sized arrays with N being an input of the program, I double-checked the function and it is working correctly. My constraints are given by:
def Brhs(mins): # Constraint over Bj
return np.sum(mins.reshape(4,N)[0]) - Bi
def Lrhs(mins): # Constraint over Lj
return np.sum(mins.reshape(4,N)[1]) - Li
cons = [{'type': 'eq', 'fun': lambda Bj: 1.0*Brhs(Bj)},
{'type': 'eq', 'fun': lambda Lj: 1.0*Lrhs(Lj)}]
In such a way that the sum of all components of Bj must be equal to Bi (same thing with Lj). The bounds of the variables are given by:
bounds = [(0,None)]*2*N + [(0,np.pi/2)]*2*N
For the problem to be reproducible, it's important to use the following inputs:
# Inputs:
gamma = 17.
C = 85.
T = C
Li = 1.
Bi = 0.5
N = 3
For the minimization, I'm using the cyipopt library (that is just similar to the scipy.optimize). Then, the minimization is given by:
from cyipopt import minimize_ipopt
x0 = np.ones(4*N) # Initial guess
res = minimize_ipopt(Fnret, x0, constraints=cons, bounds=bounds)
The problem is that the result is not obeying the conditions I imposed on the constraints (i.e. the sum of Bj or Lj values is different than Bi or Li on the results). But, for instance, if I only write one of the two constraints (over Lj or Bj) it works fine for that variable. Maybe I'm missing something when using 2 constraints and I can't find the error, it seems that it doesn't work with both constraints together. Any help would be truly appreciated. Thank you in advance!
P.S.: In addition, I would like that the function result F to be positive as well. How can I impose this condition? Thanks!
Not a complete answer, just some hints in arbitrary order:
Your initial point x0 is not feasible because it contradicts both of your constraints. This can easily be observed by evaluating your constraints at x0. Under the hood, Ipopt typically detects this and tries to find a feasible initial point. However, it's highly recommended to provide a feasible initial point whenever possible.
Your variable bounds are not well-defined. Evaluating your objective at your bounds yields multiple divisions by zero. For example, the denominator np.tan(b[i]) is zero if and only if b[i] = 0, so 0 isn't a valid value for all of your b[i]s. Proceeding similarly for the other terms, you should obtain 0 < b[i] < pi/2 and 0 ≤ a[i] < pi/2. Here, you can model the strict inequalities by 0 + eps ≤ b[i] ≤ pi/2 - eps and 0 ≤ a[i] ≤ pi/2 - eps, where eps is a sufficiently small positive number.
If you really want to impose that the objective function is always positive, you can simply add the inequality constraint Fnret(x) >= 0, i.e. {'type': 'ineq', 'fun': Fnret}.
In Code:
# bounds
eps = 1e-8
bounds = [(0, None)]*2*N + [(0, np.pi/2 - eps)]*N + [(0+eps, np.pi/2 - eps)]*N
# (feasible) initial guess
x0 = eps*np.ones(4*N)
x0[[0, N]] = [Bi-(N-1)*eps, Li-(N-1)*eps]
# constraints
cons = [{'type': 'eq', 'fun': Brhs},
{'type': 'eq', 'fun': Lrhs},
{'type': 'ineq', 'fun': Fnret}]
res = minimize_ipopt(Fnret, x0, constraints=cons, bounds=bounds, options={'disp': 5})
Last but not least, this still doesn't converge to a stationary point, so chances are that there's indeed no local minimum. From here, you can try experimenting with other (feasible!) initial points and double-check the math of your problem. It's also worth providing the exact gradient and constraint Jacobians.
So, based on #joni suggestions, I could find a stationary point respecting the constraints by adopting the trust-constr method of scipy.optimize.minimize library. My code is running as follows:
import numpy as np
from scipy.optimize import minimize
# Inputs:
gamma = 17
C = 85.
T = C
Li = 2.
Bi = 1.
N = 3 # for instance
# Constraints:
def Brhs(mins):
return np.sum(mins.reshape(4,N)[0]) - Bi/2
def Lrhs(mins):
return np.sum(mins.reshape(4,N)[1]) - Li/2
# Function to minimize:
def Fnret(mins):
Bj, Lj, a, b = mins.reshape(4,N)
S1 = 0; S2 = 0
Binf = np.zeros(N); Linf = np.zeros(N);
for i in range(N):
sbi=(Bi/2); sli=(Li/2)
for j in range(i+1):
sbi -= Bj[j]
sli -= Lj[j]
Binf[i]=sbi
Linf[i]=sli
for i in range(N):
S1 += (C*(1-np.sin(a[i]))+T*np.sin(a[i])) * ((2*Bj[i]*Binf[i]+Bj[i]**2)/(np.tan(b[i])*np.cos(a[i]))) +\
(C*(1-np.sin(b[i]))+T*np.sin(b[i])) * ((2*Bj[i]*Linf[i]+Lj[i]*Bj[i])/(np.sin(b[i])))
S2 += (gamma*Bj[0]/(6*np.tan(b[0])))*((Bi/2)*(Li/2) + 4*(Binf[0]+Bj[0])*(Linf[0]+Lj[0]) + Binf[0]*Linf[0])
j=1
while j<(N):
S2 += (gamma*Bj[j]/(6*np.tan(b[j])))*(Binf[j-1]*Linf[j-1] + 4*(Binf[j]+Bj[j])*(Linf[j]+Lj[j]) + Binf[j]*Linf[j])
j += 1
F = 2*(S1+S2)
return F
eps = 1e-3
bounds = [(0,None)]*2*N + [(0+eps,np.pi/2-eps)]*2*N # Bounds
cons = ({'type': 'ineq', 'fun': Fnret},
{'type': 'eq', 'fun': Lrhs},
{'type': 'eq', 'fun': Brhs})
x0 = np.ones(4*N) # Initial guess
res = minimize(Fnret, x0, method='trust-constr', bounds = bounds, constraints=cons, tol=1e-6)
F = res.fun
Bj = (res.x).reshape(4,N)[0]
Lj = (res.x).reshape(4,N)[1]
ai = (res.x).reshape(4,N)[2]
bi = (res.x).reshape(4,N)[3]
Which is essentially the same just changing the minimization technique. From np.sum(Bj) and np.sum(Lj) is easy to see that the results are in agreement with the constraints imposed, which were not working previously.

Minimization of a function with iterative bounds in Python

I'm trying to minimize a function of N parameters (e.g. x[1],x[2],x[3]...,x[N]) where the boundaries for the minimization depend on the minimized parameters themselves. For instance, assume that all values of x could vary between 0 and 1 in such a way that the summing then all I got 1, then I have the following inequalities for the boundaries:
0 <= x[1] <= 1
x[1] <= x[2] <= 1 - x[1]
x[2] <= x[3] <= 1-x[1]-x[2]
...
x[N-1] <= x[N] <= 1-x[1]-x[2]-x[3]-...-x[N]
Does anyone have an idea on how can I construct some algorithm like that on python? Or maybe if I can adopt an existent method from Scipy for example?
As a rule of thumb: As soon as your boundaries depend on the optimization variables, they are inequality constraints instead of boundaries. Using 0-based indices, your inequalities can be written as
# left-hand sides
-x[0] <= 0
x[i] - x[i+1] <= 0 for all i = 0, ..., n-1
# right-hand sides
sum(x[i], i = 0, .., j) - 1 <= 0 for all j = 0, .., n
Both can be expressed by a simple matrix-vector product:
import numpy as np
D_lhs = np.diag(np.ones(N-1), k=-1) - np.diag(np.ones(N))
D_rhs = np.tril(np.ones(N))
def lhs(x):
return D_lhs # x
def rhs(x):
return D_rhs # x - np.ones(x.size)
As a result, you can use scipy.optimize.minimize to minimize your objective function subject to lhs(x) <= 0 and rhs(x) <= 0 like this:
from scipy.optimize import minimize
# minmize expects eqach inequality constraint in the form con(x) >= 0,
# so lhs(x) <= 0 is the same as -1.0*lhs(x) >= 0
con1 = {'type': 'ineq', 'fun': lambda x: -1.0*lhs(x)}
con2 = {'type': 'ineq', 'fun': lambda x: -1.0*rhs(x)}
result = minimize(your_obj_fun, x0=inital_guess, constraints=(con1, con2))

Inconsistency in solutions using CVXPY

Please, consider the following optimisation problem. Specifically, x and b are (1,n) vectors, C is (n,n) symmetric matrix, k is an arbitrary constant and i is a (1,n) vector of ones.
Please, also consider the following equivalent optimisation problem. In such case, k is determined during the optimisation process so there is no need to scale the values in x to obtain the solution y.
Please, also consider the following code for solving both the problems with cvxpy.
import cvxpy as cp
import numpy as np
def problem_1(C):
n, t = np.shape(C)
x = cp.Variable(n)
b = np.array([1 / n] * n)
obj = cp.quad_form(x, C)
constraints = [b.T # cp.log(x)>=0.5, x >= 0]
cp.Problem(cp.Minimize(obj), constraints).solve()
return (x.value / (np.ones(n).T # x.value))
def problem_2(C):
n, t = np.shape(C)
y = cp.Variable(n)
k = cp.Variable()
b = np.array([1 / n] * n)
obj = cp.quad_form(y, C)
constraints = [b.T # cp.log(y)>=k, np.ones(n)#y.T==1, y >= 0]
cp.Problem(cp.Minimize(obj), constraints).solve()
return y.value
While the first function do provide me with the correct solution for a sample set of data I am using, the second does not. Specifically, values in y differ heavily while employing the second function with some of them being equal to zero (which cannot be since all values in b are positive and greater than zero). I am wondering wether or not the second function minimise also k. Its value should not be minimised on the contrary it should just be determined during the optimisation problem as the one that leads to the solution that minimise the objective function.
UPDATE_1
I just found that the solution that I obtain with the second formulation of the problem is equal to the one derived with the following equations and function. It appears that the constraint with the logarithmic barrier and the k variable is ignored.
def problem_3(C):
n, t = np.shape(C)
y = cp.Variable(n)
k = cp.Variable()
b = np.array([1 / n] * n)
obj = cp.quad_form(y, C)
constraints = [np.ones(n)#y.T==1, y >= 0]
cp.Problem(cp.Minimize(obj), constraints).solve()
return y.value
UPDATE_2
Here is the link to a sample input C - https://www.dropbox.com/s/kaa7voufzk5k9qt/matrix_.csv?dl=0. In such case the correct output for both problem_1 and problem_2 is approximately equal to [0.0659 0.068 0.0371 0.1188 0.1647 0.3387 0.1315 0.0311 0.0441] since they are equivalent by definition. I am able to obtain the the correct output by solving only problem_1. Solving problem_2 leads to [0.0227 0. 0. 0.3095 0.3392 0.3286 0. 0. 0. ] which is wrong since it happens to be the correct output for problem_3.
UPDATE_3
To be clear, by definition problem_2 exhibits solution equal to the solution of problem_3 when the parameter k goes to minus infinity.
UPDATE_4
Please consider the following code that is for solving problem_1 using SciPy Optimize instead CVXPY. By imposing k=9 the correct optimal solution can still be achieved which is consistent with problem_1 being independent of the parameter.
import scipy.optimize as opt
def obj(x, C):
return x.T # C # x
def problem_1_1(C):
n, t = np.shape(C)
b = np.array([1 / n] * n)
constraints = [{"type": "eq", "fun": lambda x: (b * np.log(x)).sum() - 9}]
res = opt.minimize(
obj,
x0 = np.array([1 / n] * n),
args = (C),
bounds = ((0, None),) * n,
constraints = constraints
)
return (res['x'] / (np.ones(n).T # res['x']))
UPDATE_5
By considering the code in UPDATE_4, whenever k is set equal to 10 the correct solution is still achieved however appears the following warning. I suppose that is due to rounding error that might occur during the optimisation process.
Untitled.py:56: RuntimeWarning: divide by zero encountered in
log {"type": "eq", "fun": lambda x: (b * np.log(x)).sum() - 10}
I am wondering if there is a way to impose strict inequality constraint with CVXPY or apply a condition on the logarithm argument. Please consider the following modified code for problem_1_1.
import scipy.optimize as opt
def obj(x, C):
return x.T # C # x
def problem_1_1(C):
n, t = np.shape(C)
b = np.array([1 / n] * n)
constraints = [{"type": "eq", "fun": lambda x: (b * np.log(x if x.all() > 0 else 1e-100)).sum() - 10}]
res = opt.minimize(
obj,
x0 = np.array([1 / n] * n),
args = (C),
bounds = ((0, None),) * n,
constraints = constraints
)
return (res['x'] / (np.ones(n).T # res['x']))
UPDATE_6
To be thorough, the correct value of optimal k is approximatively -2.4827186402337564.
If you let be arbitrary then you are basically saying that is greater or equal to some arbitrary number, which is trivially true, so the constraint becomes irrelevant.
I believe you should either fix the value of or turn this problem into a minimax problem by determining a tadeoff betweenmaximizing and minimizing .

Symbolic matrix and numpy usage error "TypeError: ufunc 'isfinite' not supported for the input types.."

I was trying to perform a scipy.opimization using minimize function. I am looking to find all the variables like Iz,Iy,J,kz,ky,Yc,Yg such that the error between vector K_P_X and f is minimum. That is objective function K_P_X-fshould be minimum. I think my mistake is related to the calculation involving numpy.linalg.norm(sol-f)where the sol is assigned with a symbolic vector (K_P_X). Due to the data type conflict i am getting this error. If that's the case, Q1. Can anyone please suggest a better way to represent the equality constraint equation (ie. constr1()) such that this error can be avoided. The full code is given below,
import scipy.optimize as optimize
from sympy import symbols,zeros,Matrix,Transpose
import numpy
#Symobolic K matrix
Zc,Yc,Zg,Yg=symbols("Zc Yc Zg Yg",real=True)
A,Iz,Iy,J,kz,ky,E,G,L=symbols("A Iz Iy J kz ky E G L",real=True,positive=True)
E=10400000;G=3909800;L=5
def phi_z():
phi_z=(12*E*Iy)/(kz*A*G*L**2)
return phi_z
def phi_y():
phi_y=(12*E*Iz)/(ky*A*G*L**2)
return phi_y
K_P=zeros(12,12)
K1=Matrix(([E*A/L,0,0],[0,(12*E*Iz)/((1+phi_y())*L**3),0],[0,0,(12*E*Iy)/((1+phi_z())*L**3)]))
K2=Matrix(([G*J/L,0,0],[0,E*Iy/L,0],[0,0,E*Iz/L]))
Q1=Matrix(([0,Zg,-Yg],[-Zc,0,L/2],[Yc,-L/2,0]))
Q1_T=Transpose(Q1)
Q2=Matrix(([0,Zg,-Yg],[-Zc,0,-L/2],[Yc,L/2,0]))
Q2_T=Transpose(Q2)
K11=K1; K12=K1*Q1; K13=-K1; K14=-K1*Q2; K22=Q1_T*K1*Q1+K2; K23=-Q1_T*K1; K24=-Q1_T*K1*Q2-K2; K33=K1; K34=K1*Q2; K44=Q2_T*K1*Q2+K2
K_P[0:3,0:3]=K11; K_P[0:3,3:6]=K12; K_P[0:3,6:9]=K13; K_P[0:3,9:12]=K14; K_P[3:6,3:6]=K22; K_P[3:6,6:9]=K23; K_P[3:6,9:12]=K24 ;K_P[6:9,6:9]=K33; K_P[6:9,9:12]=K34; K_P[9:12,9:12]=K44
##Converting Upper triangular stiffness matrix to Symmetric stiffness matrix##
for i in range(0,12):
for j in range(0,12):
K_P[j,i]=K_P[i,j]
K_P = K_P.subs({A: 7.55})
K_P = K_P.subs({Zc: 0})
K_P = K_P.subs({Zg: 0})
X= numpy.matrix([[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1],[1]])
K_P_X=K_P*X
f= numpy.matrix([[-9346.76033789],[1595512.77906],[-1596283.83112],[274222.872543],[4234010.18889],[4255484.3549],[9346.76033789],[-1595512.77906],[1596283.83112],[-275173.513088],[3747408.91068],[3722085.0499]])
function=K_P_X-f
def Obj_func(variables):
Iz,Iy,J,kz,ky,Yc,Yg=variables
function=K_P_X-f #K_P_X matrix contains the variables like Iz,Iy,J,kz,ky,Yc,Yg.
return function
def constr1(variables):
sol = K_P_X #Here the variables are in the symbolic vector K_P_X
if numpy.allclose(sol, f):
return 0.00 #If Error is equal to zero hence required accuracy is reached. So stop optimization
else:
return numpy.linalg.norm(sol-f)
initial_guess=[10,10,10,0.1,0.1,0.001,0.001]
cons = ({'type':'eq', 'fun': constr1},{'type': 'ineq', 'fun': lambda variables: -variables[3]+1},{'type': 'ineq', 'fun': lambda variables: variables[3]-0.001},{'type': 'ineq', 'fun': lambda variables: -variables[4]+1},{'type': 'ineq', 'fun': lambda variables: variables[4]-0.001},{'type': 'ineq', 'fun': lambda variables: -variables[5]+0.5},{'type': 'ineq', 'fun': lambda variables: variables[5]-0},{'type': 'ineq', 'fun': lambda variables: -variables[6]+0.5},{'type': 'ineq', 'fun': lambda variables: variables[6]-0})
bnds = ((1, 60), (1, 60),(1, 60),(0.1, 1),(0.1, 1),(0.001, 0.5),(0.001, 0.5))
res=optimize.minimize(Obj_func,initial_guess, bounds=bnds,constraints=cons)
I'll list some of the things that are wrong here.
As hpaulj said, you can't directly pass SymPy objects to SciPy or NumPy. But you can lambdify and then use that in the minimization routine
Your minimization setup does not make sense. Minimizing a function with the constraint that that same function must be zero... this is not what constrained minimization means. Constraints are something different from the objective.
It's better to use least_squares here which is dedicated to minimizing the norm of the difference (some vector function - target vector).
With that in mind, here is your script reworked:
import scipy.optimize as optimize
from sympy import symbols, Matrix, lambdify
import numpy
Iz,Iy,J,kz,ky,Yc,Yg = symbols("Iz Iy J kz ky Yc Yg",real=True,positive=True)
K_P_X = Matrix([[37.7776503296448*Yg + 8.23411191827681],[-340.454138522391*Iz/(21.1513673253807*Iz/ky + 125)],[-9.4135635827062*Iy*Yc/(21.1513673253807*Iy/kz + 125) - 368.454956983948*Iy/(21.1513673253807*Iy/kz + 125)],[-9.4135635827062*Iy*Yc**2/(21.1513673253807*Iy/kz + 125) - 368.454956983948*Iy*Yc/(21.1513673253807*Iy/kz + 125) - 0.0589826136148473*J],[23.5339089567655*Iy*Yc/(21.1513673253807*Iy/kz + 125) + 2.62756822555969*Iy + 921.137392459871*Iy/(21.1513673253807*Iy/kz + 125)],[-5.00660515891599*Iz - 851.135346305977*Iz/(21.1513673253807*Iz/ky + 125) - 37.7776503296448*Yg**2 - 8.23411191827681*Yg],[-37.7776503296448*Yg - 8.23411191827681],[340.454138522391*Iz/(21.1513673253807*Iz/ky + 125)],[9.4135635827062*Iy*Yc/(21.1513673253807*Iy/kz + 125) + 368.454956983948*Iy/(21.1513673253807*Iy/kz + 125)],[9.4135635827062*Iy*Yc**2/(21.1513673253807*Iy/kz + 125) + 368.454956983948*Iy*Yc/(21.1513673253807*Iy/kz + 125) + 0.0589826136148473*J],[23.5339089567655*Iy*Yc/(21.1513673253807*Iy/kz + 125) - 2.62756822555969*Iy + 921.137392459871*Iy/(21.1513673253807*Iy/kz + 125)],[5.00660515891599*Iz - 851.135346305977*Iz/(21.1513673253807*Iz/ky + 125) + 37.7776503296448*Yg**2 + 8.23411191827681*Yg]])
f = Matrix([[-1],[-1],[-1],[-1.00059553353],[3.99999996539],[-5.99940443072],[1],[1],[1],[1],[1],[1]])
obj = lambdify([Iz,Iy,J,kz,ky,Yc,Yg], tuple(K_P_X - f))
initial_guess=[10,10,10,0.1,0.1,0.001,0.001]
bnds = ((1, 60), (1, 60),(1, 60),(0.1, 1),(0.1, 1),(0.001, 0.5),(0.001, 0.5))
lower = [a for (a, b) in bnds]
upper = [b for (a, b) in bnds]
res = optimize.least_squares(lambda x: obj(x[0], x[1], x[2], x[3], x[4], x[5], x[6]), initial_guess, bounds=(lower, upper))
print(res)
Changes:
Prior to lambdify, we should have a SymPy expression. So both K_P_X and f are SymPy matrices now.
Lambdified function takes 7 scalar arguments and returns a tuple of components of K_P_X - f
The bounds are separated into lower and upper, as the syntax of least_squares requires
We can't directly pass obj to least_squares, because it will receive one array parameter instead of 7 scalars. Hence the additional lambda step for unpacking the vector.
Believe it or not, minimization works. It returns res.x, the minimum point, as
[ 1.00000000e+00, 1.00000000e+00, 1.69406332e+01,
1.00000000e-01, 1.00000000e-01, 1.00000000e-03,
1.00000000e-03]
which looks suspiciously round at first, but this is only because the point hits against the bounds you placed (10, 1, 0.1 and so on). Only the third variable ended up with an inactive constaint.

Numpy Minimize COBYLA Constraints

I am using scipy.minimize with the COBYLA method, but I seem to be unable to write the constraints properly because when I check the values of the objective function, they do not respect those constraints.
Basically, the objective function accepts an array as argument, that must follow two constraints:
Each value in the array must be greater than 0
The sum of the values must be inferior to 1
So far I wrote it this way:
constraints = [{'type': 'ineq', 'fun': lambda x: 1 - sum(x)},
{'type': 'ineq', 'fun': lambda x: x[0]},
{'type': 'ineq', 'fun': lambda x: x[1]}]
However, sometimes I get values greater than 1...
Here is an example:
from __future__ import division
from math import pow, exp
import numpy as np
from scipy.optimize import minimize
nbStudy = 3
nbCYP = 2
raucObserved = [3.98, 2.0, 0.12]
IXmat = np.matrix([[-0.98, 0], [-0.3, -0.98], [7.7, 4.2]])
NBITER = 50
estimatedCR = []
raucPred = []
varR = [0.0085, 0.0048, 0.0110]
sdR = [0.0922, 0.0692, 0.1051]
cnstrts = [{'type': 'ineq', 'fun': lambda x: 1 - sum(x)},
{'type': 'ineq', 'fun': lambda x: x}]
def fun(CR):
dum = []
for i in range(nbStudy):
crix = 0
for j in range(nbCYP):
crix += CR[j] * IXmat[i, j]
raucPredicted = 1 / (1 + crix)
dum.append(pow((np.log(raucPredicted) - np.log(raucObservedBiased[i])), 2) / varR[i])
output = np.sum(dum)
return output
for iter in range(NBITER):
raucObservedBiased = []
for k in range(nbStudy):
raucObservedBiased.append(raucObserved[k] * exp(sdR[k] * np.random.normal()))
initialCR = np.matrix([[(1 / nbCYP) * np.random.uniform()], [(1 / nbCYP) * np.random.uniform()]])
output = minimize(fun, initialCR, method='COBYLA', constraints=cnstrts)
estimatedCR.append(output.x)
Apparently a version problem, the issue has been fixed since. I was using Python 2.7 and Scipy 0.13...
You are not checking that the solver converged (output.success == True) --- and in your case it does not converge. If there is no convergence, nothing is guaranteed about the constraints.

Categories

Resources