Develop Reference

How to define the objective function for integer optimization task?

I need to find the k in the range [1, 10], which is the least positive integer such that binomial(k, 2)≥ m, where m≥3 - integer. The binomial() function is the binominal coefficient.
My attempt is:
After some algebraic steps, I have found the minization task: min k(k-1) - 2m ≥ 0, s.t. m≥3. I have defined the objective function and gradient. In the objective function I fixed the m=3 and my problem is how to define integer domain for the variable m.
from scipy.optimize import line_search
# objective function
def objective(k):
m = 3
return k*(k-1)-2*m
# gradient for the objective function
def gradient(k):
return 2.0 * k - 1
# define range
r_min, r_max = 1, 11
# prepare inputs
inputs = arange(r_min, r_max, 1)
# compute targets
targets = [objective(k) for k in inputs]
# define the starting point
point = 1.0
# define the direction to move
direction = 1.0
# print the initial conditions
print('start=%.1f, direction=%.1f' % (point, direction))
# perform the line search
result = line_search(objective, gradient, point, direction)
print(result)
I have see the
LineSearchWarning: The line search algorithm did not converge
Question. How to define the objective function in Python?

You are look to minimise k such that k(k-1)-2m≥0, with additional constraints on k on which we'll come back later. You can explicitly solve this inequation, by solving the corresponding equation first, that is, finding the roots of P:=X²-X-2m. The quadratic formulas give the roots (1±√(1+4m²))/2. Since P(x)→∞ as x→±∞, you know that the x that satisfy your inequation are the ones above the greatest root, and below the lowest root. Since you are only interested in positive solutions, and since 1-√(1+m²)<0, the set of wanted solutions is [(1+√(1+m²))/2,∞). Among these solutions, the smallest integer is the ceil of (1+√(1+m²))/2 which is strictly greater than 1. Let k=ceil((1+sqrt(1+m**2))/2) be that integer. If k≤10, then your problem has a solution, which is k. Otherwise, your problem has no solutions. In Python, you get the following:
import math
def solve(m):
k = math.ceil((1+math.sqrt(1+m**2))/2)
return k if k <= 10 else None

cvxpy -> OSQP or cvxpy -> CVXOPT how does it work under the hood?

I have the following simple program:
import numpy as np
import cvxpy as cp
np.random.seed(0)
n = 100
i = 20
y = np.random.rand(n)
A = np.random.rand(i, n).T
x = cp.Variable(n)
lmbd = cp.Variable(i)
objective = cp.Minimize(cp.sum_squares(x - y))
constraints = [x == A*lmbd,
lmbd >= np.zeros(i),
cp.sum(lmbd) == 1]
prob = cp.Problem(objective, constraints)
result = prob.solve(verbose=True)
I would like to know what happens under the hood. I know that for example the solver OSQP is being used thanks to the following variable prob.solver_stats.solver_name, I might also decide to use another solver (e.g. result = prob.solve(solver="CVXOPT", verbose=True)).
I would like to know how the problem is treated. I have the idea that it should be pre-treated since it seems like a double problem (the quadratic minimization one - y variable, and the lmbd variable as a constraint satisfaction). However, in the CVXOPT documentation, it seems to me, that the problem should be only treated as a quadratic or linear problem. In the case of CVXOPT, I know how to use it, and I wouldn't know how to translate the problem in this case, however, CVXPY does this with no trouble.
Thanks for the insight.

Since your problem is a least squares, its matrices are probably only cast from least squares to quadratic programming then passed to the QP solver as is. (This operation is simpler than the SOCP→QP conversion mentioned in the comments.)

Global minimization of multivariable with scipy.optimize.brute

I'm trying to minimize the following function:
with respect to the parameters H and alpha using the brute force method, specifically the scipy.optimize.brute algorithm. The problem arises that I don't know how to deal with this unknown number of variables, I mean, that is 2n variables and n is an input of the program.
I have the following code, where I'd like that the minimization would lead to arrays for Hand alpha values:
import numpy as np
#Entries:
gamma = 17.0
C = 70.0
T = 1
R = 0.5
n = int(2)
def F(mins, gamma,C,T,R):
H,alpha = mins
ret = 0
for i in range(n):
inner_sum = 0
for j in range(i+1):
inner_sum += H[j]*np.tan(alpha[j])
ret += 3*gamma*H[i]*(R+inner_sum)**2
So I can get the values of H and alpha from the position of the array. I was used to multivariable minimization with brute force but only when I have a fixed number of variables. In this case, how can I proceed?
P.S.: I know that the minimization of the above expression will lead to 0 for both variables. This is just a small piece of a bigger expression to illustrate the problem, in which a working algorithm would be very helpful. Thanks in advance!

Nonlinear constraints with scipy

The problem at hand is optimization of multivariate function with nonlinear constraints.
There is a differential equation (in its oversimplified form)
dy/dx = y(x)*t(x) + g(x)
I need to minimize the solution of the DE y(x), but by varying the t(x).
Since it is physics under the hood, there are constraints on t(x). I successfully implemented all of them except one:
0 < t(x) < 1 for any x in range [a,b]
For certainty, the t(x) is a general polynomial:
t(x) = a0 + a1*x + a2*x**2 + a3*x**3 + a4*x**4 + a5*x**5
The x is fixed numpy.ndarray of floats and the optimization goes for coefficients a. I use scipy.optimize with trust-constr.
What I have tried so far:
Root finding at each step and determining the minimal/maximal value of the function using optimize.root and checking for sign changes. Return 0.5 if constraints are satisfied and numpy.inf or -1 or whatever not in [0;1] range if constraints are not satisfied. The optimizer stops soon and the function is not minimized properly.
Since x is fixed-length and known, I tried to define a constraint for each point, so I got N constraints where N = len(x). This works (at least look like) but takes forever for not-so large N. Also, since x is discrete and non-uniform, I can't be sure that there are no violated constraints for any x in [a,b].
EDIT #1: the minimal reproducible example
import scipy.optimize as optimize
from scipy.optimize import Bounds
import numpy as np
# some function y(x)
x = np.linspace(-np.pi,np.pi,100)
y = np.sin(x)
# polynomial t(z)
def t(a,z):
v = 0.0;
for ii in range(len(a)):
v += a[ii]*z**ii
return v
# let's minimize the sum
def targetFn(a):
return np.sum(y*t(a,x))
# polynomial order
polyord = 3
# simple bounds to have reliable results,
# otherwise the solution will grow toward +-infinity
bnd = 10.0
bounds = Bounds([-bnd for i in range(polyord+1)],
[bnd for i in range(polyord+1)])
res = optimize.minimize(targetFn, [1.0 for i in range(polyord+1)],
bounds = bounds)
if np.max(t(res.x,x))>200:
print('max constraint violated!')
if np.min(t(res.x,x))<-100:
print('min constraint violated!')
In the reproducible example given above, let the constraints to be that the value of the polynomial t(a,x) is in range [-100;200] for the given x.
So the question is: how does one properly define a constraint to tell the optimizer that the function's values must be constrained for the given range of arguments?

CVXPY failing randomly on basic quadratic problem

I'm finding CVXPY is randomly failing with the following error:
ArpackError: ARPACK error 3: No shifts could be applied during a cycle of the Implicitly restarted
Arnoldi iteration. One possibility is to increase the size of NCV relative to NEV.
The code below is a minimal example where it is just trying to do mean variance optimisation with no constraints, identity correlation matrix, and normally distributed mean vector. Roughly once in every thousand runs this fails. It doesn't seem to matter which solver I ask it to use, which makes me think it is failing setting up the problem?
import cvxpy as cp
import numpy as np
n = 199
np.random.seed(100)
mu = np.random.normal(size = n)
C = np.eye(n)
for repeat in range(1000):
x = cp.Variable(n)
mean = x.T # mu
variance = cp.quad_form(x, C)
objective = cp.Maximize(mean - variance)
constraints = []
prob = cp.Problem(objective, constraints)
result = prob.solve()
print(repeat, end = " ")