Develop Reference

How to obtain multiple solutions of a binary LP problem using Google's OR-Tools in Python?

I am new to integer optimization. I am trying to solve the following large (although not that large) binary linear optimization problem:
max_{x} x_1+x_2+...+x_n
subject to: A*x <= b ; x_i is binary for all i=1,...,n
As you can see,
. the control variable is a vector x of lengh, say, n=150; x_i is binary for all i=1,...,n
. I want to maximize the sum of the x_i's
. in the constraint, A is an nxn matrix and b is an nx1 vector. So I have n=150 linear inequality constraints.
I want to obtain a certain number of solutions, NS. Say, NS=100. (I know there is more than one solution, and there are potentially millions of them.)
I am using Google's OR-Tools for Python. I was able to write the problem and to obtain one solution. I have tried many different ways to obtain more solutions after that, but I just couldn't. For example:
I tried using the SCIP solver, and then I used the value of the objective function at the optimum, call it V, to add another constraint, x_1+x_2+...+x_n >= V, on top of the original "Ax<=b," and then used the CP-SAT solver to find NS feasible vectors (I followed the instructions in this guide). There is no optimization in this second step, just a quest for feasibility. This didn't work: the solver produced N replicas of the same vector. Still, when asked for the number of solutions found, it misleadingly replies that solution_printer.solution_count() is equal to NS. Here's a snippet of the code that I used:
# Define the constraints (A and b are lists)
for j in range(n):
constraint_expr = [int(A[j][l])*x[l] for l in range(n)]
model.Add(sum(constraint_expr) <= int(b[j][0]))
V = 112
constraint_obj_val = [-x[l] for l in range(n)]
model.Add(sum(constraint_obj_val) <= -V)
# Call the solver:
solver = cp_model.CpSolver()
solution_printer = VarArraySolutionPrinterWithLimit(x, NS)
solver.parameters.enumerate_all_solutions = True
status = solver.Solve(model, solution_printer)
I tried using the SCIP solver and then using solver.NextSolution(), but every time I was using this command, the algorithm would produce a vector that was less and less optimal every time: the first one corresponded to a value of, say, V=112 (the optimal one!); the second vector corresponded to a value of 111; the third one, to 108; fourth to sixth, to 103; etc.
My question is, unfortunately, a bit vague, but here it goes: what's the best way to obtain more than one solution to my optimization problem?
Please let me know if I'm not being clear enough or if you need more/other chunks of the code, etc. This is my first time posting a question here :)
Thanks in advance.

Is your matrix A integral ? if not, you are not solving the same problem with scip and CP-SAT.
Furthermore, why use scip? You should solve both part with the same solver.
Furthermore, I believe the default solution pool implementation in scip will return all solutions found, in reverse order, thus in decreasing quality order.

In Gurobi, you can do something like this to get more than one optimal solution :
solver->SetSolverSpecificParametersAsString("PoolSearchMode=2"); // or-tools [Gurobi]
From Gurobi Reference [Section 20.1]:
By default, the Gurobi MIP solver will try to find one proven optimal solution to your model.
You can use the PoolSearchMode parameter to control the approach used to find solutions.
In its default setting (0), the MIP search simply aims to find one
optimal solution. Setting the parameter to 1 causes the MIP search to
expend additional effort to find more solutions, but in a
non-systematic way. You will get more solutions, but not necessarily
the best solutions. Setting the parameter to 2 causes the MIP to do a
systematic search for the n best solutions. For both non-default
settings, the PoolSolutions parameter sets the target for the number
of solutions to find.
Another way to find multiple optimal solutions could be to first solve the original problem to optimality and then add the objective function as a constraint with lower and upper bound as the optimal objective value.

Pyomo warm start

I've a MIP to solve with Pyomo and I want to set an initial solution for cplex.
So googling I find that I can set some variable of instance to some value and then execute this:
solver.solve(instance,warmstart=True,tee=True)
But when I run cplex it seem that it doesn't use the warm start, because for example i pass a solution with value 16, but in 5 seconds it return a solution with value 60.
So I don't know there is some error or other stuff that doesn't work.
P.S.
I don't know if is a problem but my warm start solution set only some variale to a value, but not all. Could be a problem?

Make sure that the solution you give to CPLEX is feasible. Otherwise, CPLEX will reject it and start from scratch.
If your solution is feasible, it is possible that CPLEX simply found a better solution than yours, since, after all, it is CPLEX's job, and in my own experience, CPLEX is very good at it. Is this a maximization problem? If so, in your example, CPLEX found a better solution (objective=60) than yours (objective=16), which is the expected behavior.
Sadly, CPLEX is often greedy in term of verbose, so it is hard to know from the solver log if warmstart was used or not (unlike its competitor GUROBI where it is clearly written in the log). However, it seems like you started the warmstart correctly, using the warmstart=True parameter.
If, however, your problem isn't a maximization problem, it is possible that CPLEX will not make a differenciation between the variables that you gave a value and the variable that still holds a solution from last solve. Plus, giving values to only a fraction of your variables might make the problem infeasible, considering that all values not manually specified are the values previously found by CPLEX. ex: contraint x<=2y. The solver found x=2, y=1 as a feasible solution. You define x:=3, then your constraint is not respected (y is still =1 for CPLEX, so the constraint x<=2y is 3<=2, which is false). CPLEX will see it as infeasible and will reject your solution.
One alternative that I can give you, if you absolutely want to use your own values in the final solution, is instead of defining values for your variables, create a constraint that explicitly defines your variable value. This constraint can afterward be "deactivated" if needed. But be careful, as this does not necessarily yield the optimal solution, but the "optimal solution when some variables have the specific value".

Using Pyomo with heuristic solvers

I am using Pyomo to model my optimization problem (MILP) and solve it using Gurobi.
What would be the best, fastest or easiest way to find a heuristic solution using the Pyomo model, knowing that I do not care about the Gap bounds.
Note: I know that Gurobi has a heuristic solver but it doesn't tell what heuristic algorithm they are using!

Finding a heuristic solution to some MILP problem is complexity-wise as hard as optimizing it!
There is no best, fastest, easiest way in general. You always want to exploit some problem-characteristics.
As start, just use any MIP-solver and tune the params to reflect your needs. If you want just any heuristic solution, tune the solver for feasibility, probably meaning a higher frequency of heuristic-steps and early-stop with the first feasible solution.
Yes, you won't know what's Gurobi using internally. But knowing all of the code would not help much either. It's surely not something which you can find on wikipedia then (except for classic stuff like the feasibility pump or Relaxation induced neighborhood search).
If you want to know more about these methods, check out papers on MIP-heuristics in general! You will see, that most Heuristics are tightly coupled with the MIP-nature of the problem (although i expect some SAT-solver-usage internally too in commercial ones).

Computational cost of scipy.newton_krylov

I am trying to compare the computational cost of several methods of solving a nonlinear partial differential equation. One of the methods uses scipy.optimize.newton_krylov. The other methods use scipy.sparese.linalg.lgmres.
My first thought was to count the number of iterations of newton_krylov using the 'callback' keyword. This worked, to a point. The counter that I set up counts the number of iterations made by the newton_krylov solver.
I would also like to count the number of steps it takes in each (Newton) iteration to solve the linear system. I thought that I could use a similar counter and try using a keyword like 'inner_callback', but that gave the number of iterations the newton_krylov solver used rather than the number of steps to solve the inner linear system.
It is possible that I don't understand how newton_krylov was implemented and that it is impossible to separate the the two types of iterations (Newton steps vs lgmres steps) but it would be great if there was a way.
I tried posting this on scicomp.stackexchange, but I didn't get any answers. I think that the question might be too specific to Scipy to get a good answer there, so I hope that someone here can help me. Thank you for your help!

0/1 Knapsack with few variables: which algorithm?

I have to implement the solution to a 0/1 Knapsack problem with constraints.
My problem will have in most cases few variables (~ 10-20, at most 50).
I recall from university that there are a number of algorithms that in many cases perform better than brute force (I'm thinking, for example, to a branch and bound algorithm).
Since my problem is relative small, I'm wondering if there is an appreciable advantange in terms of efficiency when using a sophisticate solution as opposed to brute force.
If it helps, I'm programming in Python.

You can either use pseudopolynomial algorithm, which uses dynamic programming, if the sum of weights is small enough. You just calculate, whether you can get weight X with first Y items for each X and Y.
This runs in time O(NS), where N is number of items and S is sum of weights.
Another possibility is to use meet-in-the middle approach.
Partition items into two halves and:
For the first half take every possible combination of items (there are 2^(N/2) possible combinations in each half) and store its weight in some set.
For the second half take every possible combination of items and check whether there is a combination in first half with suitable weight.
This should run in O(2^(N/2)) time.

Brute force stuff would work fine for 10 variables, but for, say, 40 you'd get some 1000'000'000'000 possible solutions, which would probably take too long to enumerate. I'd consider approximate algorithms, e.g. the polynomial time algorithm (see, e.g. http://math.mit.edu/~goemans/18434S06/knapsack-katherine.pdf) or use a search algorithm such as branch-and-bound, maybe with an additional heuristic.

Brute force algorithms will always return the best solutions. The problem with them is that in exponential order problems they quickly become not feasible.
If you are guaranteed to have up to 20 variables, you will test no more than 1 million solutions (2^20= 1M). Hence, brute force is feasible and no other algorithm will return a better solution.
Heuristics are great, but they should be used only when we have no exact solution to the problem. There is a great book that might help you: How to Solve it, by Michalewicz.