Develop Reference

Conditional summing in Pyomo

I'm trying to reduce the number of binary variables I need in a Big-M reformulation from a linear number to a logarithmic one. I have the following equation:
image link (for i = 1, ..., m)
Here, hi are known vectors where each element has a binary value. zj are unknown binary variables.
So, if hi = z the constraint is enforced.
What I was wondering is, is there a way to code these conditional sums in Pyomo?
I know that in Mosel we can use the '|' operator to add conditions but was unable to find something similar for Pyomo. Any suggestions are greatly appreciated.

There are probably a couple ways to do this, but the below gets the job done w/ no additional variables and just a little "examination" of h inside of the constraint construction. Bringing h "inside" the model is optional, but I think good practice.
Code
# big M with vector
# making a big-M constraint from a vector matching operation
import pyomo.environ as pyo
big_m = 11
h = [[1, 0, 1], [0, 0, 1]]
m = pyo.ConcreteModel()
m.I = pyo.Set(initialize=[0, 1])
m.J = pyo.Set(initialize=[0, 1, 2])
m.z = pyo.Var(m.J, domain=pyo.Binary)
m.x = pyo.Var()
m.h = pyo.Param(m.I, domain=pyo.Any, initialize=h)
m.A = pyo.Param(m.I, initialize=[2, 3])
m.b = pyo.Param(m.I, initialize=[4, 5])
#m.Constraint(m.I)
def big_m(m, i):
# make a couple subsets of J on-the-fly
ones = {idx for idx, value in enumerate(m.h[i]) if value==1}
zeros = m.J.difference(ones)
# build the "enforcement" summation
enforce = len(ones) - sum(m.z[j] for j in ones) + sum(m.z[j] for j in zeros)
# use it
return m.A[i] * m.x <= m.b[i] + enforce * big_m
m.big_m.pprint()
# set some values to see the constraint evaluated... (simulate solving)
m.z.set_values({0:1, 1:0, 2:1})
m.x.set_value(1.5)
m.big_m.display()
Output:
big_m : Size=2, Index=I, Active=True
Key : Lower : Body : Upper : Active
0 : -Inf : 2*x - (4 + (2 - (z[0] + z[2]) + z[1])*11) : 0.0 : True
1 : -Inf : 3*x - (5 + (1 - z[2] + z[0] + z[1])*11) : 0.0 : True
big_m : Size=2
Key : Lower : Body : Upper
0 : None : -1.0 : 0.0
1 : None : -11.5 : 0.0

In Pyomo, you can implement this constraint using a conditional expression in a Constraint object.
Here how you can do it :
from pyomo.environ import *
m = ... # the number of elements in hi
model = ConcreteModel() model.hi = Var(range(m), within=Binary)
model.z = Var(range(m), within=Binary)
def conditional_sum(model, i):
return (model.hi[i] == 1) <= (model.z[i] == 1)
model.con = Constraint(range(m), rule=conditional_sum)

Pyomo: How to include a penalty in the objective function

I'm trying to minimize the cost of manufacturing a product with two machines. The cost of machine A is $30/product and cost of machine B is $40/product.
There are two constraints:
we must cover a demand of 50 products per month (x+y >= 50)
the cheap machine (A) can only manufacture 40 products per month (x<=40)
So I created the following Pyomo code:
from pyomo.environ import *
model = ConcreteModel()
model.x = Var(domain=NonNegativeReals)
model.y = Var(domain=NonNegativeReals)
def production_cost(m):
return 30*m.x + 40*m.y
# Objective
model.mycost = Objective(expr = production_cost, sense=minimize)
# Constraints
model.demand = Constraint(expr = model.x + model.y >= 50)
model.maxA = Constraint(expr = model.x <= 40)
# Let's solve it
results = SolverFactory('glpk').solve(model)
# Display the solution
print('Cost=', model.mycost())
print('x=', model.x())
print('y=', model.y())
It works ok, with the obvious solution x=40;y=10 (Cost = 1600)
However, if we start to use the machine B, there will be a fixed penalty of $300 over the cost.
I tried with
def production_cost(m):
if (m.y > 0):
return 30*m.x + 40*m.y + 300
else:
return 30*m.x + 40*m.y
But I get the following error message
Rule failed when generating expression for Objective mycost with index
None: PyomoException: Cannot convert non-constant Pyomo expression (0 <
y) to bool. This error is usually caused by using a Var, unit, or mutable
Param in a Boolean context such as an "if" statement, or when checking
container membership or equality. For example,
>>> m.x = Var() >>> if m.x >= 1: ... pass
and
>>> m.y = Var() >>> if m.y in [m.x, m.y]: ... pass
would both cause this exception.
I do not how to implement the condition to include the penalty into the objective function through the Pyomo code.

Since m.y is a Var, you cannot use the if statement with it. You can always use a binary variable using the Big M approach as Airsquid said it. This approach is usually not recommended, since it turns the problem from LP into a MILP, but it is effective.
You just need to create a new Binary Var:
model.bin_y = Var(domain=Binary)
Then constraint model.y to be zero if model.bin_y is zero, or else, be any value between its bounds. I use a bound of 100 here, but you can even use the demand:
model.bin_y_cons = Constraint(expr= model.y <= model.bin_y*100)
then, in your objective just apply the new fixed value of 300:
def production_cost(m):
return 30*m.x + 40*m.y + 300*model.bin_y
model.mycost = Objective(rule=production_cost, sense=minimize)

PuLP: How to write a multi variable constraint?

I am trying to solve this optimization problem in Python. I have written the following code using PuLP:
import pulp
D = range(0, 10)
F = range(0, 10)
x = pulp.LpVariable.dicts("x", (D), 0, 1, pulp.LpInteger)
y = pulp.LpVariable.dicts("y", (F, D), 0, 1, pulp.LpInteger)
model = pulp.LpProblem("Scheduling", pulp.LpMaximize)
model += pulp.lpSum(x[d] for d in D)
for f in F:
model += pulp.lpSum(y[f][d] for d in D) == 1
for d in D:
model += x[d]*pulp.lpSum(y[f][d] for f in F) == 0
model.solve()
The one-but-last line here returns: TypeError: Non-constant expressions cannot be multiplied. I understand it is returning this since it cannot solve non-linear optimization problems. Is it possible to formulate this problem as a proper linear problem, such that it can be solved using PuLP?

It is always a good idea to start with a mathematical model. You have:
min sum(d, x[d])
sum(d,y[f,d]) = 1 ∀f
x[d]*sum(f,y[f,d]) = 0 ∀d
x[d],y[f,d] ∈ {0,1}
The last constraint is non-linear (it is quadratic). This can not be handled by PuLP. The constraint can be interpreted as:
x[d] = 0 or sum(f,y[f,d]) = 0 ∀d
or
x[d] = 1 ==> sum(f,y[f,d]) = 0 ∀d
This can be linearized as:
sum(f,y[f,d]) <= (1-x[d])*M
where M = |F| (number of elements in F, i.e. |F|=10). You can check that:
x[d]=0 => sum(f,y[f,d]) <= M (i.e. non-binding)
x[d]=1 => sum(f,y[f,d]) <= 0 (i.e. zero)
So you need to replace your quadratic constraint with this linear one.
Note that this is not the only formulation. You could also linearize the individual terms z[f,d]=x[d]*y[f,d]. I'll leave that as an exercise.

Optimize Python math code for fixed values of variables in function

I have a very long math formula (just to put you in context: it has 293095 characters) which in practice will be the body of a python function. This function has 15 input parameters as in:
def math_func(t,X,P,n1,n2,R,r):
x,y,z = X
a,b,c = P
u1,v1,w1 = n1
u2,v2,w2 = n2
return <long math formula>
The formula uses simple math operations + - * ** / and one function call to arctan. Here an extract of it:
r*((-16*(r**6*t*u1**6 - 6*r**6*u1**5*u2 - 15*r**6*t*u1**4*u2**2 +
20*r**6*u1**3*u2**3 + 15*r**6*t*u1**2*u2**4 - 6*r**6*u1*u2**5 -
r**6*t*u2**6 + 3*r**6*t*u1**4*v1**2 - 12*r**6*u1**3*u2*v1**2 -
18*r**6*t*u1**2*u2**2*v1**2 + 12*r**6*u1*u2**3*v1**2 +
3*r**6*t*u2**4*v1**2 + 3*r**6*t*u1**2*v1**4 - 6*r**6*u1*u2*v1**4 -
3*r**6*t*u2**2*v1**4 + r**6*t*v1**6 - 6*r**6*u1**4*v1*v2 -
24*r**6*t*u1**3*u2*v1*v2 + 36*r**6*u1**2*u2**2*v1*v2 +
24*r**6*t*u1*u2**3*v1*v2 - 6*r**6*u2**4*v1*v2 -
12*r**6*u1**2*v1**3*v2 - 24*r**6*t*u1*u2*v1**3*v2 +
12*r**6*u2**2*v1**3*v2 - 6*r**6*v1**5*v2 - 3*r**6*t*u1**4*v2**2 + ...
Now the point is that in practice the bulk evaluation of this function will be done for fixed values of P,n1,n2,R and r which reduces the set of free variables to only four, and "in theory" the formula with less parameters should be faster.
So the question is: How can I implement this optimization in Python?
I know I can put everything in a string and do some sort of replace,compile and eval like in
formula = formula.replace('r','1').replace('R','2')....
code = compile(formula,'formula-name','eval')
math_func = lambda t,x,y,z: eval(code)
It would be good if some operations (like power) are substituted by their value, for example 18*r**6*t*u1**2*u2**2*v1**2 should become 18*t for r=u1=u2=v1=1. I think compile should do so but in any case I'm not sure. Does compile actually perform this optimization?
My solution speeds up the computation but if I can squeeze it more it will be great. Note: preferable within standard Python (I could try Cython later).
In general I'm interesting in a pythonic way to accomplish my goal maybe with some extra libraries: what is a reasonably good way of doing this? Is my solution a good approach?
EDIT: (To give more context)
The huge expression is the output of a symbolic line integral over an arc of circle. The arc is given in space by the radius r, two ortho-normal vectors (like the x and y axis in a 2D version) n1=(u1,v1,w1),n2=(u2,v2,w2) and the center P=(a,b,c). The rest is the point over which I'm performing the integration X=(x,y,z) and a parameter R for the function I'm integrating.
Sympy and Maple just take ages to compute this, the actual output is from Mathematica.
If you are curious about the formula here it is (pseudo-pseudo-code):
G(u) = P + r*(1-u**2)/(1+u**2)*n1 + r*2*u/(1+u**2)*n2
integral of (1-|X-G(t)|^2/R^2)^3 over t

You could use Sympy:
>>> from sympy import symbols
>>> x,y,z,a,b,c,u1,v1,w1,u2,v2,w2,t,r = symbols("x,y,z,a,b,c,u1,v1,w1,u2,v2,w2,t,r")
>>> r=u1=u2=v1=1
>>> a = 18*r**6*t*u1**2*u2**2*v1**2
>>> a
18*t
Then you can create a Python function like this:
>>> from sympy import lambdify
>>> f = lambdify(t, a)
>>> f(1)
18
And that f function is indeed simply 18*t:
>>> import dis
>>> dis.dis(f)
1 0 LOAD_CONST 1 (18)
3 LOAD_FAST 0 (_Dummy_18)
6 BINARY_MULTIPLY
7 RETURN_VALUE
If you want to compile the resulting code into machine code, you can try a JIT compiler such as Numba, Theano, or Parakeet.

Here's how I would approach this problem:
compile() your function to an AST (Abstract Syntax Tree) instead of a normal bytecode function - see the standard ast module for details.
Traverse the AST, replacing all references to the fixed parameters with their fixed value. There are libraries such as macropy that may be useful for this, I don't have any specific recommendation.
Traverse the AST again, performing whatever optimizations this might enable, such as Mult(1, X) => X. You don't have to worry about operations between two constants, as Python (since 2.6) optimizes that already.
compile() the AST into a normal function. Call it, and hope that the speed was increased by a sufficient amount to justify all the pre-optimization.
Note that Python will never optimize things like 1*X on its own, as it cannot know what type X will be at runtime - it could be an instance of a class that implements the multiplication operation in an arbitrary way, so the result is not necessarily X. Only your knowledge that all the variables are ordinary numbers, obeying the usual rules of arithmetic, makes this optimization valid.

The "right way" to solve a problem like this is one or more of:
Find a more efficient formulation
Symbolically simplify and reduce terms
Use vectorization (e.g. NumPy)
Punt to low-level libraries that are already optimized (e.g. in languages like C or Fortran that implicitly do strong expression optimization, rather than Python, which does nada).
Let's say for a moment, though, that approaches 1, 3, and 4 are not available, and you have to do this in Python. Then simplifying and "hoisting" common subexpressions is your primary tool.
The good news is, there are a lot of opportunities. The expression r**6, for example, is repeated 26 times. You could save 25 computations by simply assigning r_6 = r ** 6 once, then replacing r**6 every time it occurs.
When you start looking for common expressions here, you'll find them everywhere. It'd be nice to mechanize that process, right? In general, that requires a full expression parser (e.g. from the ast module) and is an exponential-time optimization problem. But your expression is a bit of a special case. While long and varied, it's not especially complicated. It has few internal parenthetical groupings, so we can get away with a quicker and dirtier approach.
Before the how, the resulting code is:
sa = r**6 # 26 occurrences
sb = u1**2 # 5 occurrences
sc = u2**2 # 5 occurrences
sd = v1**2 # 5 occurrences
se = u1**4 # 4 occurrences
sf = u2**3 # 3 occurrences
sg = u1**3 # 3 occurrences
sh = v1**4 # 3 occurrences
si = u2**4 # 3 occurrences
sj = v1**3 # 3 occurrences
sk = v2**2 # 1 occurrence
sl = v1**6 # 1 occurrence
sm = v1**5 # 1 occurrence
sn = u1**6 # 1 occurrence
so = u1**5 # 1 occurrence
sp = u2**6 # 1 occurrence
sq = u2**5 # 1 occurrence
sr = 6*sa # 6 occurrences
ss = 3*sa # 5 occurrences
st = ss*t # 5 occurrences
su = 12*sa # 4 occurrences
sv = sa*t # 3 occurrences
sw = v1*v2 # 5 occurrences
sx = sj*v2 # 3 occurrences
sy = 24*sv # 3 occurrences
sz = 15*sv # 2 occurrences
sA = sr*u1 # 2 occurrences
sB = sy*u1 # 2 occurrences
sC = sb*sc # 2 occurrences
sD = st*se # 2 occurrences
# revised formula
sv*sn - sr*so*u2 - sz*se*sc +
20*sa*sg*sf + sz*sb*si - sA*sq -
sv*sp + sD*sd - su*sg*u2*sd -
18*sv*sC*sd + su*u1*sf*sd +
st*si*sd + st*sb*sh - sA*u2*sh -
st*sc*sh + sv*sl - sr*se*sw -
sy*sg*u2*sw + 36*sa*sC*sw +
sB*sf*sw - sr*si*sw -
su*sb*sx - sB*u2*sx +
su*sc*sx - sr*sm*v2 - sD*sk
That avoids 81 computations. It's just a rough cut. Even the result could be further improved. The subexpressions sr*sw and su*sd for example, could be pre-computed as well. But we'll leave that next level for another day.
Note that this doesn't include the starting r*((-16*(. The majority of the simplification can be (and needs to be) done on the core of the expression, not on its outer terms. So I stripped those away for now; they can be added back once the common core is computed.
How do you do this?
f = """
r**6*t*u1**6 - 6*r**6*u1**5*u2 - 15*r**6*t*u1**4*u2**2 +
20*r**6*u1**3*u2**3 + 15*r**6*t*u1**2*u2**4 - 6*r**6*u1*u2**5 -
r**6*t*u2**6 + 3*r**6*t*u1**4*v1**2 - 12*r**6*u1**3*u2*v1**2 -
18*r**6*t*u1**2*u2**2*v1**2 + 12*r**6*u1*u2**3*v1**2 +
3*r**6*t*u2**4*v1**2 + 3*r**6*t*u1**2*v1**4 - 6*r**6*u1*u2*v1**4 -
3*r**6*t*u2**2*v1**4 + r**6*t*v1**6 - 6*r**6*u1**4*v1*v2 -
24*r**6*t*u1**3*u2*v1*v2 + 36*r**6*u1**2*u2**2*v1*v2 +
24*r**6*t*u1*u2**3*v1*v2 - 6*r**6*u2**4*v1*v2 -
12*r**6*u1**2*v1**3*v2 - 24*r**6*t*u1*u2*v1**3*v2 +
12*r**6*u2**2*v1**3*v2 - 6*r**6*v1**5*v2 - 3*r**6*t*u1**4*v2**2
""".strip()
from collections import Counter
import re
expre = re.compile('(?<!\w)\w+\*\*\d+')
multre = re.compile('(?<!\w)\w+\*\w+')
expr_saved = 0
stmts = []
secache = {}
seindex = 0
def subexpr(e):
global seindex
cached = secache.get(e)
if cached:
return cached
base = ord('a') if seindex < 26 else ord('A') - 26
name = 's' + chr(seindex + base)
seindex += 1
secache[e] = name
return name
def hoist(e, flat, c):
"""
Hoist the expression e into name defined by flat.
c is the count of how many times seen in incoming
formula.
"""
global expr_saved
assign = "{} = {}".format(flat, e)
s = "{:30} # {} occurrence{}".format(assign, c, '' if c == 1 else 's')
stmts.append(s)
print "{} needless computations quashed with {}".format(c-1, flat)
expr_saved += c - 1
def common_exp(form):
"""
Replace ALL exponentiation operations with a hoisted
sub-expression.
"""
# find the exponentiation operations
exponents = re.findall(expre, form)
# find and count exponentiation operations
expcount = Counter(re.findall(expre, form))
# for each exponentiation, create a hoisted sub-expression
for e, c in expcount.most_common():
hoist(e, subexpr(e), c)
# replace all exponentiation operations with their sub-expressions
form = re.sub(expre, lambda x: subexpr(x.group(0)), form)
return form
def common_mult(f):
"""
Replace multiplication operations with a hoisted
sub-expression if they occur > 1 time. Also, only
replaces one sub-expression at a time (the most common)
because it may affect further expressions
"""
mults = re.findall(multre, f)
for e, c in Counter(mults).most_common():
# unlike exponents, only replace if >1 occurrence
if c == 1:
return f
# occurs >1 time, so hoist
hoist(e, subexpr(e), c)
# replace in loop and return
return re.sub('(?<!\w)' + re.escape(e), subexpr(e), f)
# return f.replace(e, flat(e))
return f
# fix all exponents
form = common_exp(f)
# fix selected multiplies
prev = form
while True:
form = common_mult(form)
if form == prev:
# have converged; no more replacements possible
break
prev = form
print "--"
mults = re.split(r'\s*[+-]\s*', form)
smults = ['*'.join(sorted(terms.split('*'))) for terms in mults]
print smults
# print the hoisted statements and the revised expression
print '\n'.join(stmts)
print
print "# revised formula"
print form
Parsing with regular expressions is dicey business. That journey is prone to error, sorrow, and regret. I guarded against bad outcomes by hoisting some exponentiations that didn't strictly need to be, and by plugging random values into both the before and after formulas to make sure they both give the same results. I recommend the "punt to C" strategy if this is production code. But if you can't...

Dynamically build a lambda function in python

Supose that I want to generate a function to be later incorporated in a set of equations to be solved with scipy nsolve function. I want to create a function like this:
xi + xi+1 + xi+3 = 1
in which the number of variables will be dependent on the number of components. For example, if I have 2 components:
f = lambda x: x[0] + x[1] - 1
for 3:
f = lambda x: x[0] + x[1] + x[2] - 1
I specify the components as an array within the arguments of the function to be called:
def my_func(components):
for component in components:
.....
.....
return f
I can't just find a way of doing this. I've to be able to make it this way as this function and other functions need to be solved together with nsolve:
x0 = scipy.optimize.fsolve(f, [0, 0, 0, 0 ....])
Any help would be appreciated
Thanks!
Since I'm not sure which is the best way of doing this I will fully explain what I'm trying to do:
-I'm trying to generate this two functions to be later nsolved:
So I want to create a function teste([list of components]) that can return me this two equations (Psat(T) is a function I can call depending on the component and P is a constant(value = 760)).
Example:
teste(['Benzene','Toluene'])
would return:
xBenzene + xToluene = 1
xBenzenePsat('Benzene') + xToluenePsat('Toluene') = 760
in the case of calling:
teste(['Benzene','Toluene','Cumene'])
it would return:
xBenzene + xToluene + xCumene = 1
xBenzenePsat('Benzene') + xToluenePsat('Toluene') + xCumene*Psat('Cumene') = 760
All these x values are not something I can calculate and turn into a list I can sum. They are variables that are created as a function ofthe number of components I have in the system...
Hope this helps to find the best way of doing this

A direct translation would be:
f = lambda *x: sum(x) - 1
But not sure if that's really what you want.

You can dynamically build a lambda with a string then parse it with the eval function like this:
a = [1, 2, 3]
s = "lambda x: "
s += " + ".join(["x[" + str(i) + "]" for i in xrange(0, 3)]) # Specify any range
s += " - 1"
print s
f = eval(s)
print f(a)

I would take advantage of numpy and do something like:
def teste(molecules):
P = np.array([Psat(molecule) for molecule in molecules])
f1 = lambda x: np.sum(x) - 1
f2 = lambda x: np.dot(x, P) - 760
return f1, f2
Actually what you are trying to solve is a possibly underdetermined system of linear equations, of the form A.x = b. You can construct A and b as follows:
A = np.vstack((np.ones((len(molecules),)),
[Psat(molecule) for molecule in molecules]))
b = np.array([1, 760])
And you could then create a single lambda function returning a 2 element vector as:
return lambda x: np.dot(A, x) - b
But I really don´t think that is the best approach to solving your equations: either you have a single solution you can get with np.linalg.solve(A, b), or you have a linear system with infinitely many solutions, in which case what you want to find is a base of the solution space, not a single point in that space, which is what you will get from a numerical solver that takes a function as input.

If you really want to define a function by building it up iteratively, you can. I can't think of any situation where this would be the best answer, or even a reasonable one, but it's what you asked for, so:
def my_func(components):
f = lambda x: -1
for component in components:
def wrap(f):
return lambda x: component * x[0] + f(x[1:])
f = wrap(f)
return f
Now:
>>> f = my_func([1, 2, 3])
>>> f([4,5,6])
44
Of course this will be no fun to debug. For example, look at the traceback from calling f([4,5]).

def make_constraint_function(components):
def constraint(vector):
return sum(vector[component] for component in components) - 1
return constraint
You could do it with a lambda, but a named function may be more readable. deffed functions can do anything lambdas can and more. Make sure to give the function a good docstring, and use variable and function names appropriate for your program.