I am trying to minimize a function of a vector of length 20, but I want to constrain the solution to be monotonic, i.e.
x[1] <= x[2]... <= x[20]
I have tried to implement this in the following way using "constraints" for this routine:
cons = tuple([{'type':'ineq', 'fun': lambda x: x[i]- x[i-1]} for i in range(1, len(node_vals))])
res = sp.optimize.minimize(localisation, b, args=(d), constraints = cons) #optimize
However, the results I get are not monotonic, even when the initial guess b is, it seems that the optimizer is completely ignoring the constraints. What could be going wrong? I have also tried changing the constraint to x[i]**3 - x[i+1]**3 to make it "smoother", but it didn't help at all. My objective function, localisation is the integral of solution to an eigenvalue problem whose parameters are defined beforehand:
def localisation(node_vals, domain): #calculate localisation for solutions with piecewise linear grading
f = piecewise(node_vals, domain) #create piecewise linear function using given values at nodes
#plt.plot(domain, f(domain))
M = diff_matrix(f(domain)) #differentiation matrix created from piecewise linear function
m = np.concatenate(([0], get_solutions(M)[1][:, 0], [0]))
integral = num_int(domain, m)
return integral
You didn’t post a minimum reproducible example that we can run. However, did you try to specify which optimization algorithm to use in SciPy? Something like this:
res = sp.optimize.minimize(localisation, b, args=(d), constraints = cons, method=‘SLSQP’)
I'm having a very similar problem but with additional upper and lower bounds on the monotonicity property. I'm tackling the problem like this (maybe it helps you):
Using the Trust-Region Constrained Algorithm given by scipy. This provides us a way of dealing with linear constraints in a matrix-manner:
lb <= A.dot(x) <= ub
where lb & and ub are the lower (upper) bounds of this constraint problem and A is the matrix, representing the linear constraint problem.
every row of matrix A is a linear term which defines a constraint
If, for example, x[0] <= x[1], then this can be transformed into x[0] - x[1] <= 0 which in terms of the linear constraint matrix A looks like this [1, -1,...], provided that the upper bound vector has a 0 value on this level of course (vice versa is also possible but either way, having at least one of both, lower or upper bound, makes this easy)
Setting up enough of these inequalities and at the same time merging a couple of those into a single inequality may create a sufficient matrix to solve this.
Hope this helps a bit, It did the job for my problem.
Related
I want to solve the following (convex) minimization problem:
min ||x||_1 under the constraints sgn(A[x,R]=y) and ||x||_2 = 1
where A is a mx(N+1) matrix, x in R^N a vector, and \[x,R\] a vector that is created by appending a given number R. The objective is to find the optimal value for x.
A is a Fourier matrix and there are fast matrix-vector, inversion, etc. algorithms available. Since this matrix is really big, I need to use an optimization algorithm that utilizes this.
Currently, I use the following implementation in cvxpy, which is way too slow:
import cvxpy as cvx
# rewrite the problem in the form x = x^- + x^+
n = A.shape[1]-1
vx = cvx.Variable(2*n)
objective = cvx.Minimize(cvx.pnorm(vx, 1)) # min ||x||_1
constraints = [vx >= 0, cvx.multiply(A[:,:n] # vx[:n] - A[:,:n] # vx[n:] + A[:,n]*R, y) >= 0,
cvx.norm(vx, 2) <= R] # sgn(A[x,1]) = y, ||x||_2 <= R
x, solve_time = solve(vx, objective, constraints)
solution = x[:n] - x[n:]
Is there a way to use fast matrix computations in cvxpy? Or is there a better library? I found a few implementations that can do this for one special algorithm but not in the general case, so I was not able to implement my problem.
No. The solver will not call your matrix multiplication code. They do their own linear algebra, which is very different in many ways. In a sense your matrix multiplication is just notation for the problem statement.
Regarding performance, it depends heavily on where the bottleneck is. Is it in generating the model (in cvxpy itself) or in the solver? What solver are you using? Consider using a different solver. Obviously, we don't have enough information (and no reproducible example) to answer this question.
Suppose I have an objective function f(x) that is non-convex (x can be a vector), but once I add constraints it becomes a convex problem. To show what i mean, consider this trivial example: let f(x) = cos(x). Clearly, cos(x) is not convex, but if i only consider x in [0, pi/2], then the function is convex when restricting x to these values.
CVXPY does not accept such a problem because it does not satisfy DCP rules. One option, in the previous example, is to minimize f(x) + g(x) where g is an indicator function such that: g(x) = 0 for x in [0, pi/2] and g(x) = +infinity otherwise, but I don't know how to implement that in CVXPY. What can i use in Python to take advantage of the "convexity" of the problem?
Thanks.
I would like to know how to define a complex objective function using or-tools (if it is possible).
The basic example below shows how to have basic linear problem with Or-tools in python:
solver = pywraplp.Solver('lp_pricing_problem', pywraplp.Solver.GLOP_LINEAR_PROGRAMMING)
# Define variables with a range from 0 to 1000.
x = solver.NumVar(0, 1000, 'Variable_x')
y = solver.NumVar(0, 1000, 'Variable_y')
# Define some constraints.
solver.Add(x >= 17)
solver.Add(x <= 147)
solver.Add(y >= 61)
solver.Add(y <= 93)
# Minimize 0.5*x + 2*y
objective = solver.Objective()
objective.SetCoefficient(x, 0.5)
objective.SetCoefficient(y, 2)
objective.SetMinimization()
status = solver.Solve()
# Print the solution
if status == solver.OPTIMAL:
print("x: {}, y: {}".format(x.solution_value(), y.solution_value())) # x: 17.0, y: 61.0
In this very basic example the objective function is Minimize(0.5*x + 2*y).
What would be the syntax to obtain, for example, the least squares Minimize(x^2 + y^2) or the absolute value of a variable Minimize(abs(x) + y)?
Is it possible to define a sub-function and call it into the objective function? Or should I proceed another way?
Many thanks in advance,
Romain
You've tagged this question with linear-programming, so you already have the ingredients to figure out the answer here.
If you check out this page, you'll see that OR-Tools solves linear programs, as well as few other families of optimization problems.
So the first objective function you mention, Minimize(0.5*x + 2*y) is solvable because it is linear.
The second objective you mention---Minimize(x^2 + y^2)---cannot be solved with OR-Tools because it is nonlinear: those squared terms make it quadratic. To solve this problem you need something that can do quadratic programming, second-order cone programming, or quadratically constrained quadratic programming. All of these methods include linear programming as a subset. The tool I recommend for solving these sorts of problems is cvxpy, which offers a powerful and elegant interface. (Alternatively, you can approximate the quadratic as linear-piecewise, but you will incur many more constraints.)
The last objective you mention, Minimize(c*abs(x) + y) can be solved as a linear program even though abs(x) itself is nonlinear. To do so, we rewrite the objective as min( c*(t1-t2) +y) and add the constraints t1,t2>=0. This works as long as c is positive and you are minimizing (or c is negative and you are maximizing). A longer explanation is here.
There are many such transformations you can perform and one of the skills of a mathematical programmer/operations researcher is to have many of them memorized.
I have been trying to find an answer to this problem for a couple of hours now, but i can't find anything so far...
So I have two vectors let's call them b and x, of which i know all values. They add up to be the same amount, so sum(b) = sum(x).
I also have a Matrix, let's call it A, of which i know what values are 0, all the other values are unknown (but are different from 0).
Furthermore, the the elements of each column of A has the sum of 1 (I think that's called it's a left stochastic matrix)
Generally the Equation can be written in the form A*x = b.
Now I'm trying to find the missing values of A.
I have found one answer to the general problem here: https://math.stackexchange.com/questions/1170843/solving-ax-b-when-x-and-b-are-given
Furthermore i looked at the documentation of numpy.linalg
:https://docs.scipy.org/doc/numpy/reference/routines.linalg.html, but i just can't figure out how to do it.
It looks similar to a multi linear regression problem, but also on sklearn, i couldn't find anything: https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html#sklearn.linear_model.LinearRegression
Not a complete answer, but a bit of a more formal statement of the problem.
I think this can be solved as just a system of linear equations. Let
NZ = {(i,j)|a(i,j) is not fixed to zero}
Then write:
sum( j | (i,j) ∈ NZ, a(i,j) * x(j) ) = b(i) ∀i
sum( i | (i,j) ∈ NZ, a(i,j)) = 1 ∀j
This is just a system of linear equations in a(i,j). It may be under- (or over-) determined and it may be sparse. I think it depends a bit on this how to solve it. It may possible to think about these as constraints in a linear (or quadratic) programming problem. That would allow you to add an objective (in case of an underdetermined system or overdetermined -- in that case minimize sum of squared deviations, or 1-norm of deviations). In addition we can add bounds on a(i,j) (e.g. lower bounds of zero and upper bounds of one). So a linear programming approach may be what you are looking for.
This problem looks a bit like matrix balancing. This is used a lot for economic data sets that come from different sources and where we want to reconcile the data to get a consistent data set usable for subsequent modeling.
I am trying to solve an ODE using Python's solve_ivp. However, I want to change the right hand side of my ODE dynamically based on a comparison between the current solution and previous solution. The idea behind this is that my right hand side is a vector field, and I want to ensure directionality of the vector field by reversing the right hand side based on the direction of the previous solution.
The implementation for this is as follows: I want to check the dot product in the right hand side function definition between the previous solution and the vector field. If the dot product is negative the right hand side is multiplied by -1.
I therefore need to access the previous state of the ODE solver and use it in comparison with the current iteration. In MATLAB there is the possibility of using "OutputFcn" while solving an ODE. This function is called after every iteration of the integrator. In the function it is therefore possible to simply extract the state as a variable and use it in the next iteration. I have not been able to find something similar for Python.
def RHS(timesnotused,x):
out = solve_ivp(doubleGyreVar, [0,T/2, T], [x[0], x[1], 1, 0, 0, 1], rtol = 1e-10, atol=1e-10)
output = out.y
J = output[2:,-1].reshape(2,2)
CG = np.matmul(J.T , J)
lambdas, xis = np.linalg.eig(CG)
xi_1 = xis[np.argmin(lambdas)]
xi_2 = xis[np.argmax(lambdas)]
lambda_1 = np.min(lambdas)
lambda_2 = np.max(lambdas)
alpha = ((lambda_2-lambda_1) / (lambda_2+lambda_1))**2
sign = 1
if np.dot(xlast,xi_1) < 0:
sign = -1
return(sign*alpha*xi_1)
As can be seen I want "xlast" to be the previous solution, and check it with xi_1 of the current iteration. Somehow xlast needs to be updated every iteration.
If the differential equation you want to solve is
dx/dt = sign(g(x)) * F(x)
for some sufficiently smooth functions g, F, then you have a discontinuous right side where all advanced numerical solvers will produce nonsense as soon as they approach this jump singularity.
The clearest method to solve such a multi-phase system is to present only continuous right sides to the numerical solver and to handle the phase change via the event mechanism that is also present in scipy.integrate.solve_ivp.
I explored a mechanism to do that with the tools of scipy in a similar problem with a sign function producing a discontinuity in odeint returns wrong results for an ODE including descrete function