As part of a BuildAction rule that gets triggered on creation of a concrete model, I am dynamically creating additional "internal" decision variables (dependent on data supplied at construction time).
As well as creating these variables (which get used in constraint expressions), I know that I also need to add them to the model to avoid the "Variable 'XXX' is not part of the model being written out, but appears in an expression used on this model." error.
The VarList class seems designed for this (by analogy to the ConstraintList class which I am already successfully using for dynamically created constraints). However, I cannot find documentation for how to populate a VarList from pre-created variables. I can create a VarList and add variables to it, but this does not give me the control I need over how the variables are created...
import pyomo.environ as pyo
self.vl = pyo.VarList()
newVar = self.vl.add() # this does not give me control over the variable creation
# and I can't set all required properties of newVar, once created
It seems that I should be able to create a VarList by passing a dictionary of variables, but I cannot find documentation or examples that show how this works.
VarList works pretty similar to an IndexedVar in Pyomo. You need to understand a couple of things:
The variable index is constantly changing. This means that you need to check the actual length to avoid adding variables that you won't use, or to use variable that have not been added.
The VarList().add() method adds variables of the same type as in VarList(). For example, if VarList() was created as an Integer or NonNegativeReal variable, all variables you will add will be Integer or NonNegativeReal, respectively.
Here's example to show you how it works:
import pyomo.environ as pyo
# Create the model
model = pyo.ConcreteModel()
# Add variables in a loop
model.x = pyo.VarList(domain=pyo.Integers)
for i in range(2):
model.x.add() # Add a new index to defined variable x
# Adding constraints
# Indexed variable starts at 1 and not in 0
model.myCons1 = pyo.Constraint(expr=2*model.x[1] + 0.5*model.x[2] <=20)
model.myCons2 = pyo.Constraint(expr=2+model.x[1] + 3*model.x[2] <=25)
# Add an objective
model.Obj = pyo.Objective(expr=model.x[1] + model.x[2], sense=pyo.maximize)
# Solve using Gurobi
solver = pyo.SolverFactory('gurobi')
solver.solve(model, tee=True)
# Display the x variable results
model.x.display()
This leads to the following output (showing only the relevant part):
Optimal solution found (tolerance 1.00e-04)
Best objective 1.300000000000e+01, best bound 1.300000000000e+01, gap 0.0000%
x : Size=2, Index=x_index
Key : Lower : Value : Upper : Fixed : Stale : Domain
1 : None : 8.0 : None : False : False : Integers
2 : None : 5.0 : None : False : False : Integers
If you start using the kernel modeling layer you can also use the pyomo.kernel.variable_list class, which works pretty similar to this approach. You can check it in the Pyomo documentation with the difference that you can assign different types of variables to the same list.
I don't fully understand what you are modeling, but you can always use an AbstractModel() class and then populate it with some external data (from a .dat file or a dict, etc.) using model.create_instance(data=data). In this way, your model is always parameterized by some defined sets.
Related
I am currently working on a MILP formulation that I want to solve using Gurobi with a branch-and-cut approach. My model is a variation of a classic Pickup and Delivery Problem with Time Windows (PDPTW), for which several classes of valid inequalities are defined. As the branch-and-bound solver runs, I want to add those inequalities (i.e., I want to add cuts), if certain conditions in the current node are met. My issue is as follows:
My variables are defined as dictionaries, which makes it easy to use them when formulating constraints because I can easily use their original indexing. An example of how I define variables is provided below
tauOD = {}
# Start- End-Service time of trucks
for i in range(0,Nt):
tauOD[i,0]=model.addVar(lb=0.0, ub=truckODTime[i][0],
vtype=GRB.CONTINUOUS,name='tauOD[%s,%s]'%(i,0))
tauOD[i,1]=model.addVar(lb=0.0, ub=truckODTime[i][1],
vtype=GRB.CONTINUOUS,name='tauOD[%s,%s]'%(i,1))
Once my model is defined in terms of variables, constraints, and cost function, in a classic branch-and-bound problem I would simply use model.optimize() to start the process. In this case, I am using the command model.optimize(my_callback), where my_callback is the callback function I defined to add cuts. My issue is that the callback function, for some reasons, does not like model variables defined as dictionaries. The only workaround I found is as follows:
model._vars = model.getVars() #---> added this call right before the optimization starts
model.optimize(mycallback)
and then inside the callback I can now retrieve variables using their ordering, not their indices as follows:
def mycallback(model,where):
if where == GRB.Callback.MIPNODE:
status = model.cbGet(GRB.Callback.MIPNODE_STATUS)
# If current node was solved to optimality, add cuts to strenghten
# linear relaxation
if status == GRB.OPTIMAL:
this_Sol = model.cbGetNodeRel(model._vars) # Get variables of current solution
# Adding a cut
model.cbCut(lhs=this_Sol[123]+this_Sol[125],sense=GRB.LESS_EQUAL,rhs=1) #---> Dummy cut just
# for illustration
# purposes
The aforementioned cut is just a dummy example to show that I can add cuts using the order variables are sequenced in my solution, and not their indexing. As example, I would like to be able to write a constraint inside my callback as
x[0,3,0] + x[0,5,0] <= 1
but the only thing I can do is to write
this_Sol[123] + this_Sol[125] <= 1 (assuming x[0,3,0] is the 124-th variable of my solution vector, and x[0,5,0] is the 126-th). Although knowing the order of variables is doable, because it depends on how I create them when setting up the model, it is a much more challenging process (and error-prone) rather than being able to use the indices, as I do when defining the original constraints of my model (see below for an example):
###################
### CONSTRAINTS ###
###################
# For each truck, one active connection from origin depot
for i in range(0,Nt):
thisLHS = LinExpr()
for j in range(0,sigma):
thisLHS += x[0,j+1,i]
thisLHS += x[0,2*sigma+1,i]
model.addConstr(lhs=thisLHS, sense=GRB.EQUAL, rhs=1,
name='C1_'+str(i))
Did any of you experience a similar problem? A friend of mine told me that Gurobi, for some reasons, does not like variables defined as dictionaries inside a callback function, but I do not know how to circumvent this.
Any help would be greatly appreciated.
Thanks!
Alessandro
You should make a copy of the variables by their dicts.
To get the variable index, you also have to make a copy of the lists os indexes.
Try this:
model._I = model.I
model._J = model.J
model._K = model.K
model._x = model.x
You need theses indexes lists so you can loop each target variable x to verify some condition. As you would do writing a normal constraint for your model.
Then inside your callback you can make the index iterations:
def mycallback(model,where):
if where == GRB.Callback.MIPNODE:
x = model.cbGetSolution(model._x)
for i in model._I:
if sum([x[i,j,k] for j in model._J for k in model._K]) > 1:
Add_the_cut()
First time pyomo user here.
Is there any difference between declaring sets in an optimization model as
model.A = Set(initialize = list_values)
and using a built in python set like this one?
A = set(list_values)
Are there any advantages of one over the other?
First you can call the model variables, sets, parameters in the constraint rules via:
def some_constraint_rule(model):
return model.some_set
So you cant do something like this via python sets because the model you input in the constraint rule does not have your python set.
Second you cannot index python sets (so far I know, therefore u need a dict??), but with pyomo sets, u can do much more stuff eg.:
m.t = pyomo.Set(
within=m.index,
initialize=something,
ordered=True,
doc='some documentation for your set?')
I finished an ILP before and it works properly.
opt = SolverFactory('glpk')
model = AbstractModel()
model.obj = Objective(...)
# variables, constraints ...
instance = model.create_instance()
results = opt.solve(instance)
since I want to get the value of each variable but also the objective function solved, I try to access the Objective function by the way similar as what I did to the variable but all I can get is an expression.
I use the following code:
print(instance.obj.value)
But only got the warning like this:
WARNING: DEPRECATED: The .value property getter on SimpleObjective is deprecated. Use the .expr property getter instead
When I change the code to
print(instance.obj.expr)
All I get is an expression. So I wanna know is there any way to get the value of the objective function other than getting all the variables needed and re-calculating by myself again?
The best way to get the value of the objective function is to use the value function provided by Pyomo
print(value(instance.obj))
The expression property getter has to be called explicitly.
obj_val = instance.obj.expr()
print(obj_val)
I have a Pyomo model which has the form:
from pyomo.environ import *
from pyomo.dae import *
m = AbstractModel()
m.t = ContinuousSet(bounds=(0,120))
m.T = Param(default=120)
m.S = Var(m.t, bounds=(0,None))
m.Sdot = DerivativeVar(m.S)
m.obj = Objective(expr=m.S[120],sense=maximize)
Note that the objective m.obj relies on the parameter m.T. Attempting to run this gives the error:
TypeError: unhashable type: 'SimpleParam'
Using a value, such as expr=m.S[120] gives the error:
ValueError: Error retrieving component S[120]: The component has not been constructed.
In both cases, my goal is the same: to optimize for the largest possible value of S at the horizon.
How can I create an abstract model which expresses this?
You are hitting on two somewhat separate issues:
TypeError: unhashable type: 'SimpleParam'
Is due to a bug in Pyomo 4.3 where you cannot directly use simple Params as indexes into other components. That said, the fix for this particular problem will not fix your example model.
The trick to fixing your Objective declaration is to encapsulate the Objective expression within a rule:
def obj_rule(m):
return m.S[120]
# or better yet:
# return m.S[m.T]
# or
# return m.S[m.t.last()]
m.obj = Objective(rule=obj_rule,sense=maximize)
The problem is that when you are writing an Abstract model, each component is only being declared, but not defined. So, the Var S is declared to exist, but has not been defined (it is an empty shell with no members). This causes a problem because Python (not Pyomo) attempts to resolve the m.S[120] to a specific variable immediately before calling the Objective constructor. The use of rules (functions) in Abstract models allows you to defer the resolution of the expression until Pyomo is actually constructing the model instance. Pyomo constructs the instance components in the same order that you declared them on the Abstract model, so when it fires the obj_rule, the previous components (S, T, and t) are all constructed and S has valid members at the known points of the ContinuousSet (in this case, the bounds).
I am writing some program using python and the z3py module.
What I am trying to do is the following: I extract a constraint of an if or a while statement from a function which is located in some other file. Additionally I extract the used variables in the statement as well as their types.
As I do not want to parse the constraint by hand into a z3py friendly form, I tried to use evaluate to do this for me. Therefore I used the tip of the following page: Z3 with string expressions
Now the problem is: I do not know how the variables in the constraint are called. But it seems as I have to name the handle of each variable like the actual variable. Otherwise evaluate won't find it. My code looks like this:
solver = Solver()
# Look up the constraint:
branch = bd.getBranchNum(0)
constr = branch.code
# Create handle for each variable, depending on its type:
for k in mapper.getVariables():
var = mapper.getVariables()[k]
if k in constr:
if var.type == "intNum":
Int(k)
else:
Real(k)
# Evaluate constraint, insert the result and solve it:
f = eval(constr)
solver.insert(f)
solve(f)
As you can see I saved the variables and constraints in classes. When executing this code I get the following error:
NameError: name 'real_x' is not defined
If I do not use the looping over the variables, but instead the following code, everything works fine:
solver = Solver()
branch = bd.getBranchNum(0)
constr = branch.code
print(constr)
real_x = Real('real_x')
int_y = Int('int_y')
f = eval(constr)
print(f)
solver.insert(f)
solve(f)
The problem is: I do not know, that the variables are called "real_x" or "int_y". Furthermore I do not know how many variables there are used, which means I have to use some dynamic thing like a loop.
Now my question is: Is there a way around this? What can I do to tell python that the handles already exist, but have a different name? Or is my approach completely wrong and I have to do something totally different?
This kind of thing is almost always a bad idea (see Why eval/exec is bad for more details), but "almost always" isn't "always", and it looks like you're using a library that was specifically designed to be used this way, in which case you've found one of the exceptions.
And at first glance, it seems like you've also hit one of the rare exceptions to the Keep data out of your variable names guideline (also see Why you don't want to dynamically create variables). But you haven't.
The only reason you need these variables like real_x to exist is so that eval can see them, right? But the eval function already knows how to look for variables in a dictionary instead of in your global namespace. And it looks like what you're getting back from mapper.getVariables() is a dictionary.
So, skip that whole messy loop, and just do this:
variables = mapper.getVariables()
f = eval(constr, globals=variables)
(In earlier versions of Python, globals is a positional-only argument, so just drop the globals= if you get an error about that.)
As the documentation explains, this gives the eval function access to your actual variables, plus the ones the mapper wants to generate, and it can do all kinds of unsafe things. If you want to prevent unsafe things, do this:
variables = dict(mapper.getVariables())
variables['__builtins__'] = {}
f = eval(constr, globals=variables)