From Google's OR-Tools Linear Solver tutorial this block of code is referenced.
foods = [solver.NumVar(0.0, solver.infinity(), item[0]) for item in data]
# Create the constraints, one per nutrient.
constraints = []
for i, nutrient in enumerate(nutrients):
constraints.append(solver.Constraint(nutrient[1], solver.infinity()))
for j, item in enumerate(data):
constraints[i].SetCoefficient(foods[j], item[i + 3])
print('Number of constraints =', solver.NumConstraints())
Solver.Constraint appears to accept a lower bound and upper bound arguments. However, when this constraint is later referenced in the nested for loop and the method, .SetCoefficient is referenced, foods[j] retrieves from list, foods, a NumVar object and item[i+3] retrieves the nutrient i of food item j. But beyond here, I can't make sense of what SetCoefficient is doing and how this functions as a linear constraint.
The source code provides the following guidance:
class Constraint(object):
r"""
The class for constraints of a Mathematical Programming (MP) model.
A constraint is represented as a linear equation or inequality.
"""
thisown = property(lambda x: x.this.own(), lambda x, v: x.this.own(v), doc="The membership flag")
def __init__(self, *args, **kwargs):
raise AttributeError("No constructor defined")
__repr__ = _swig_repr
...
def SetCoefficient(self, var: "Variable", coeff: "double") -> "void":
r"""
Sets the coefficient of the variable on the constraint.
If the variable does not belong to the solver, the function just returns,
or crashes in non-opt mode.
"""
return _pywraplp.Constraint_SetCoefficient(self, var, coeff)
Define the constraints
The constraints for Stigler diet require the total amount of the nutrients provided by all foods to be at least the minimum requirement for each nutrient. Next, we write these constraints as inequalities involving the arrays data and nutrients, and the variables food[i].
First, the amount of nutrient i provided by food j per dollar is data[j][i+3] (we add 3 to the column index because the nutrient data begins in the fourth column of data.) Since the amount of money to be spent on food j is food[j], the amount of nutrient i provided by food j is ( data[j][i+3] \cdot food[j] ). Finally, since the minimum requirement for nutrient i is nutrientsi, we can write constraint i as follows: [ \sum_{j} data[j][i+3] \cdot food[j] \geq nutrientsi ;;;;; (1) ] The following code defines these constraints.
I don't understand the syntax of these constraints. A typical linear constraint might take the form of x + 2*y <= 3; however, the syntax here doesn't function that way. How should I use OR-Tools library to define constraints in this more conventional way?
Likewise, what is the usage of SetCoefficient doing and how does it establish a constraint in the form of an inequality?
Related
I have a Python script (Pulp library) for allocating funds among a number of clients depending on their current level of funding (gap/requirements) and their membership to priority groups. However, I am not receiving the expected results.
In particular, I want:
All allocations must be positive and their sum should be equal to the total available money I have.
I want to minimize the target funding gap for the most vulnerable group (group A) and then I want that the target gap % in the less vulnerable group increase of 10%: (for group B = funding gap A1.1, for group C = funding gap B1.1...).
I have tried this:
"""
DECISION VARIABLES
"""
# Create a continuous Decision Variable and Affine Expression for the amount of additional funding received by each
# project
allocation = {}
allocation_expr = LpAffineExpression()
for z in range(n):
if priority[z] == 'X' or (requirements[z] == 0 and skip_zero_requirements):
# Projects in Priority Group 'X' don't get any allocation
allocation[project_names[z]] = pulp.LpVariable(f'allocation_{project_names[z]}', lowBound=0, upBound=0)
else:
# allocation is non negative and cannot be greater than the initial gap
allocation[project_names[z]] = pulp.LpVariable(f'allocation_{project_names[z]}', lowBound=0, upBound=(gap[z]))
allocation_expr += allocation[project_names[z]]
# Create a continuous Decision Variable and Affine Expression for the maximum GAP% within each priority group
target_group_A_expr = LpAffineExpression()
target_group_A = pulp.LpVariable(f'allocation', lowBound=0 )
target_group_A_expr += target_group_A
"""
LINEAR PROGRAMMING PROBLEM
"""
# Create the linear programming problem object
lp_prob = pulp.LpProblem('Multi-Objective Optimization', pulp.LpMaximize)
"""
OBJECTIVE FUNCTIONS
"""
# Define the objective function as an LpAffineExpression
obj = LpAffineExpression()
# MAXIMIZE the portion of additional funding allocated to projects
obj += allocation_expr
# MINIMIZE the Max GAP% within each group [actually Maximizing the -(Max GAP%)]
obj += -target_group_A_expr
# Set the Objective Function
lp_prob += obj
"""
CONSTRAINTS
"""
# Additional funding allocations to individual projects must be non-negative and not greater than the project's gap
#for v in range(n):
# lp_prob += allocation[project_names[v]] <= gap[v]
# lp_prob += allocation[project_names[v]] >= 0
# The sum of allocations to individual projects cannot be greater than the additional funding
lp_prob += pulp.lpSum([allocation[project_names[u]] for u in range(n)]) <= additional_funding
# The Max GAP % within each group >= of the GAP % of all projects in the group (proxy for dynamic max calculation)
for i, (p, group) in enumerate(priority_groups.items()):
# Get the indices of the projects in the group
group_indices = priority_groups[p] #selects the indices matching with the rows of the projects belonging to that group
# Iterate over the indices of the projects in the group
for index in group:
# Create an LpAffineExpression for the GAP% of the project
project_gap_percentage = LpAffineExpression()
if requirements[index] == 0:
project_gap_percentage += 0
else:
project_gap_percentage += (gap[index] - allocation[project_names[index]]) / requirements[index]
# Add constraint to the model
lp_prob += target_group_A == (project_gap_percentage/pow(delta_gap, i))
"""
PROGRAMMING MODEL SOLVER
"""
# Solve the linear programming problem
lp_prob.solve()
delta_gap and the additional_funding are external parameters.
I receive even negative allocations and the constrains is not always meet, e.g. in group B, C I reach level of funding gap much lower than the level of group A- sometimes they randomly go to zero. How can this be possible?
I am considering to use another library, any suggestions?
I am new to Pyomo and python and am currently trying to build a MILP model. It'd be great if someone can help me out on this.
So in my production model, I have packing families,pf, and a packing line,l, can handle a packing family at a time. I also have a set of products, p.
In my model, capacity is reserved on a family level, so I want to construct sets in such a way that pf1 in set pf includes product p1,p2,p3,p4 and pf2 includes p5,p6,p7,p8. For instance, a constraint for implementing packing capacity restriction would be:
model.ct6rePackCap = ConstraintList()
for pf in model.PF for l in model.L for t in model.T for s in model.S for n in model.N:
lhs = sum(r[p,l,t,s,n] for p in model.P)
rhs = RPack[l,pf,t,s]
model.ct6reCap.add (lhs <= rhs)
Here r is the quantity of product with product index p and RPack is the capacity reserved for the packing family pf, to which each p belongs. How can I connect p and pf here such that each element of pf (pf1, pf2..) contains a set of products e.g. (pf1 = (p1,p2,p3,p4), pf2 =(p5,p6,p7,p8)) ?
I read pyomo document and read something about subset but it didn't seem like it would achieve what I want.
Thanks a lot in advance!
I am trying to simulate a battery dispatch models with charging and discharging constraints. The BESS is charging from a Solar PV system. When I run the model currently, there are some time periods when the BESS is charging and discharging at the same time. How can I add a flag such that when Charge >, Discharge =0 and vice-versa.
def market_constraintx0(model, t):
return (model.Charge[t] <= df.loc[t,'PVGeneration']*stripeff)
model.market_rulex0 = Constraint(model.T, rule=market_constraintx0)
def market_constraintx1(model, t):
return (model.Charge[t] + model.RegDown[t] <= model.ChargeMax)
model.market_rulex1 = Constraint(model.T, rule=market_constraintx1)
def market_constraintx2(model, t):
return ( model.Discharge[t] + model.RegUp[t] <= model.DischargeMax)
model.market_rulex2 = Constraint(model.T, rule=market_constraintx2)
def charge_soc(model, t):
return model.RegUp[t] + model.Discharge[t] <= model.SoC[t] * stripeff ###Battery discharge and regup capacity is limited by SOC
model.charge_soc = Constraint(model.T, rule=charge_soc)
def discharge_soc(model, t):
return model.RegDown[t] + model.Charge[t] <= (model.SoCmax - model.SoC[t])/stripeff ### Battery can be charged by the amount of capacity left to charge.
model.discharge_soc = Constraint(model.T, rule=discharge_soc)
The constraint
x >= 0 or y >= 0
is sometimes called a complementarity condition. It can also be written as:
x * y = 0
(I assume x and y are non-negative variables). There are different ways to solve this:
complementarity solver. Some solvers support this kind of constraints directly. Complementarity constraints inside a math programming model is known as MPEC (Mathematical Programming with Equilibrium Constraints). So these solvers are sometimes called MPEC solvers.
nonlinear formulation. The constraint x*y=0 is not very easy, but a global solver should be able to handle this reliably. However these solvers only handle relatively small models (compared to local solvers).
discrete formulation. Formulate the OR condition using binary variables or a SOS1 construct. This is especially useful if the rest of the model is linear.
You may want to look into pyomo.mpec. For further information see link.
If you want to stick to (mixed-integer) linear formulations, you can also look for indicator constraints, which are discussed generally in this question. Some solvers like CPLEX and Gurobi seem to have specific constraint types for indicator constraints, but I'm not familiar with how to use those within Pyomo.
In general, you can get similar functionality by using a "Big M" formulation. In your case, something like:
model.Indicator = Var(model.T, within=Binary)
model.M = Param(initialize=1000)
def charge_indicator_constraint(model, t):
return model.M * model.Indicator[t] >= model.Charge[t]
...
def discharge_indicator_constraint(model, t):
return (1 - model.M) * model.Indicator >= model.Discharge[t]
...
As the discussed in the question I linked to, picking the right value of model.M is important to keep your model formulation "tight", and in your case, you would probably tie it directly to the power rating of your BESS.
I have a list which consist of 2 attributes which are cost and rating. I need to find possible best flights which have lower cost and a higher rating. This is a multi objecting optimization problem with minimizing and maximizing objectives. How can I implement this in DEAP?
I'm struggling to implement Individual since im very new to DEAP.
# Attribute generator
toolbox.register("cost", random.randrange, NBR_ITEMS)
toolbox.register("rating", random.randrange, NBR_ITEMS)
# Structure initializers
toolbox.register("individual", tools.initRepeat, creator.Individual, toolbox.cost, toolbox.rating) #
toolbox.register("population", tools.initRepeat, list, toolbox.individual)
Maybe you could try to encode the cost and the rating using binary numbers.
For example, let's assume the most expensive plane ticket you can get is $16384, you could store that in 14 bits (2^14 = 16384) and the rating is a number from 0 to 10 so you can store that in 4 bits so in total you can store your individuals using 18 bits.
Now you need a function to decode it:
def decode_individual(individual):
decoded_individual = ['', '']
# Decode cost (14 bits)
for i in range(0, 14):
decoded_individual[0] += str(individual[i])
# Decode rating (4 bits)
for i in range(1, 3):
decoded_individual[0] += str(individual[13 + i])
return tuple(map(lambda x: int(x, 2), decoded_individual))
You need to set up your fitness functions for a multi-objective problem, i.e. you need to provide some weights for each function that is positive if you are trying to maximize the function and negative if you are trying to minimize it. In your case, I guess you are trying to maximize de rating and minimize the cost so you could set it up as follows:
creator.create('Fitness', base.Fitness, weights=(1.0, -0.5,))
creator.create('Individual', list, fitness=creator.Fitness)
Your fitness methods should return the result of the functions you are trying to maximize and minimize in the same order as specified in the weights:
def function_cost(individual):
decoded_individual = decode_individual(individual)
return decoded_individual[0]
def function_rating(individual):
decoded_individual = decode_individual(individual)
return decoded_individual[1]
def fitness(individual):
return (function_cost(individual), function_rating(individual)),
Then, instead of registering 2 fitness functions like in your example, register just one:
toolbox.register('evaluate', fitness)
Configure DEAP to use binary data:
toolbox.register('attrBinary', random.randint, 0, 1)
toolbox.register('individual', tools.initRepeat, creator.Individual, toolbox.attrBinary, n=18) # Here you need to specify the number of bits you are using
toolbox.register('population', tools.initRepeat, list, toolbox.individual)
# Register the evaluation function (was named fitness in this example)
toolbox.register('evaluate', fitness)
# Configure your mate and mutation methods, e.g.
toolbox.register('mate', tools.cxTwoPoint)
toolbox.register('mutate', tools.mutFlipBit, indpb=0.15)
Your selection method must support multi-objective problems, NSGA2 as pointed out in your question can be used:
toolbox.register('select', tools.selNSGA2)
Then run the algorithm, you can try different values for the number of individuals (population) the number of generations and the ratings of mating and mutation:
num_pop = 50
num_gen = 100
cx_prob = 0.7
mut_prob = 0.2
best = []
for gen in range(num_gen):
offspring = algorithms.varAnd(population, toolbox, cxpb=cx_prob, mutpb=mut_prob)
fits = toolbox.map(toolbox.evaluate, offspring)
for fit, ind in zip(fits, offspring):
ind.fitness.values = fit
population = toolbox.select(offspring, k=len(population))
top = tools.selBest(population, k=1)
fitness = fitness(top[0])
print(gen, fitness, decode_individual(top[0]), top[0])
best.append(fitness[0])
You may also want to display the best individuals of each generation in a graph:
x = list(range(num_gen))
plt.plot(x, best)
plt.title("Best ticket - Cost / Rating")
plt.show()
Haven't tested this myself and I got largely inspired by some exercise I did at University so hopefully it will work for you.
The following is the objective function:
The idea is that a mean-variance optimization has already been done on a universe of securities. This gives us the weights for a target portfolio. Now suppose the investor already is holding a portfolio and does not want to change their entire portfolio to the target one.
Let w_0 = [w_0(1),w_0(2),...,w_0(N)] be the initial portfolio, where w_0(i) is the fraction of the portfolio invested in
stock i = 1,...,N. Let w_t = [w_t(1), w_t(2),...,w_t(N)] be the target portfolio, i.e., the portfolio
that it is desirable to own after rebalancing. This target portfolio may be constructed using quadratic optimization techniques such as variance minimization.
The objective is to decide the final portfolio w_f = [w_f (1), w_f (2),..., w_f(N)] that satisfies the
following characteristics:
(1) The final portfolio is close to our target portfolio
(2) The number of transactions from our initial portfolio is sufficiently small
(3) The return of the final portfolio is high
(4) The final portfolio does not hold many more securities that our initial portfolio
An objective function which is to be minimized is created by summing together the characteristic terms 1 through 4.
The first term is captured by summing the absolute difference in weights from the final and the target portfolio.
The second term is captured by the sum of an indicator function multiplied by a user specified penalty. The indicator function is y_{transactions}(i) where it is 1 if the weight of security i is different in the initial portfolio and the final portfolio, and 0 otherwise.
The third term is captured by the total final portfolio return multiplied by a negative user specified penalty since the objective is minimization.
The final term is the count of assets in the final portfolio (ie. sum of an indicator function counting the number of positive weights in the final portfolio), multiplied by a user specified penalty.
Assuming that we already have the target weights as target_w how do I setup this optimization problem in docplex python library? Or if anyone is familiar with mixed integer programming in NAG it would be helpful to know how to setup such a problem there as well.
`
final_w = [0.]*n
final_w = np.array(final_w)
obj1 = np.sum(np.absolute(final_w - target_w))
pen_trans = 1.2
def ind_trans(final,inital):
list_trans = []
for i in range(len(final)):
if abs(final[i]-inital[i]) == 0:
list_trans.append(0)
else:
list_trans.append(1)
return list_trans
obj2 = pen_trans*sum(ind_trans(final_w,initial_w))
pen_returns = 0.6
returns_np = np.array(df_secs['Return'])
obj3 = (-1)*np.dot(returns_np,final_w)
pen_count = 1.
def ind_count(final):
list_count = []
for i in range(len(final)):
if final[i] == 0:
list_count.append(0)
else:
list_count.append(1)
return list_count
obj4 = sum(ind_count(final_w))
objective = obj1 + obj2 + obj3 + obj4
The main issue in your code is that final_w is not a an array of variables but an array of data. So there will be nothing to optimize. To create an array of variables in docplex you have to do something like this:
from docplex.mp.model import Model
with Model() as m:
final = m.continuous_var_list(n, 0.0, 1.0)
That creates n variables that can take values between 0 and 1. With that in hand you can start things. For example:
obj1 = m.sum(m.abs(initial[i] - final[i]) for i in range(n))
For the next objective things become harder since you need indicator constraints. To simplify definition of these constraints first define a helper variable delta that gives the absolute difference between stocks:
delta = m.continuous_var_list(n, 0.0, 1.0)
m.add_constraints(delta[i] == m.abs(initial[i] - final[i]) for i in range(n))
Next you need an indicator variable that is 1 if a transaction is required to adjust stock i:
needtrans = m.binary_var_list(n)
for i in range(n):
# If needtrans[i] is 0 then delta[i] must be 0.
# Since needtrans[i] is penalized in the objective, the solver will
# try hard to set it to 0. It will only set it to 1 if delta[i] != 0.
# That is exactly what we want
m.add_indicator(needtrans[i], delta[i] == 0, 0)
With that you can define the second objective:
obj2 = pen_trans * m.sum(needtrans)
once all objectives have been defined, you can add their sum to the model:
m.minimize(obj1 + obj2 + obj3 + obj4)
and then solve the model and display its solution:
m.solve()
print(m.solution.get_values(final))
If any of the above is not (yet) clear to you then I suggest you take a look at the many examples that ship with docplex and also at the (reference) documentation.