Develop Reference

Adding constraints to my fitting model using lmfit

I am trying to fit a complex conductivity model (the drude-smith-anderson model) using lmfit.minimize. In that fitting, I want constraints on my parameters c and c1 such that 0<c<1, -1<c1<0 and 0<1+c1-c<1. So, I am using the following code:
#reference: Juluri B.K. "Fitting Complex Metal Dielectric Functions with Differential Evolution Method". http://juluribk.com/?p=1597.
#reference: https://lmfit.github.io/lmfit-py/fitting.html
#import libraries (numdifftools needs to be installed but doesn't need to be imported)
import matplotlib.pyplot as plt
import numpy as np
import lmfit as lmf
import math as mt
#define the complex conductivity model
def model(params,w):
sigma0 = params["sigma0"].value
tau = params["tau"].value
c = params["c"].value
d = params["d"].value
c1 = params["c1"].value
druidanderson = (sigma0/(1-1j*2*mt.pi*w*tau))*(1 + c1/(1-1j*2*mt.pi*w*tau)) - sigma0*c/(1-1j*2*mt.pi*w*d*tau)
return druidanderson
#defining the complex residues (chi squared is sum of squares of residues)
def complex_residuals(params,w,exp_data):
delta = model(params,w)
residual = (abs((delta.real - exp_data.real) / exp_data.real) + abs(
(delta.imag - exp_data.imag) / exp_data.imag))
return residual
# importing data from CSV file
importpath = input("Path of CSV file: ") #Asking the location of where your data file is kept (give input in form of path\name.csv)
frequency = np.genfromtxt(rf"{importpath}",delimiter=",", usecols=(0)) #path to be changed to the file from which data is taken
conductivity = np.genfromtxt(rf"{importpath}",delimiter=",", usecols=(1)) + 1j*np.genfromtxt(rf"{importpath}",delimiter=",", usecols=(2)) #path to be changed to the file from which data is taken
frequency = frequency[np.logical_not(np.isnan(frequency))]
conductivity = conductivity[np.logical_not(np.isnan(conductivity))]
w_for_fit = frequency
eps_for_fit = conductivity
#defining the bounds and initial guesses for the fitting parameters
params = lmf.Parameters()
params.add("sigma0", value = float(input("Guess for \u03C3\u2080: ")), min =10 , max = 5000) #bounds have to be changed manually
params.add("tau", value = float(input("Guess for \u03C4: ")), min = 0.0001, max =10) #bounds have to be changed manually
params.add("c1", value = float(input("Guess for c1: ")), min = -1 , max = 0) #bounds have to be changed manually
params.add("constraint", value = float(input("Guess for constraint: ")), min = 0, max=1)
params.add("c", expr="1+c1-constraint", min = 0, max = 1) #bounds have to be changed manually
params.add("d", value = float(input("Guess for \u03C4_1/\u03C4: ")),min = 100, max = 100000) #bounds have to be changed manually
# minimizing the chi square
minimizer_results = lmf.minimize(complex_residuals, params, args=(w_for_fit, eps_for_fit), method = 'differential_evolution', strategy='best1bin',
popsize=50, tol=0.01, mutation=(0, 1), recombination=0.9, seed=None, callback=None, disp=True, polish=True, init='latinhypercube')
lmf.printfuncs.report_fit(minimizer_results, show_correl=False)
As a result for the fit, I get the following output:
sigma0: 3489.38961 (init = 1000)
tau: 1.2456e-04 (init = 0.01)
c1: -0.99816132 (init = -1)
constraint: 0.98138820 (init = 1)
c: 0.00000000 == '1+c1-constraint'
d: 7333.82306 (init = 1000)
These values don't make any sense as 1+c1-c = -0.97954952 which is not 0 and is thus invalid. How to fix this issue?

Your code is not runnable. The use of input() is sort of stunning - please do not do that. Write code that is pleasant to read and separates i/o from logic.
To make a floating point residual from a complex array, use complex_array.view(float)
Guessing any parameter value to be at or very close to its limit (here, c) is a very bad idea, likely to make the fit harder.
More to your question, you defined c as "evaluate 1+c1-constant and then apply the bounds min=0, max=1". That is literally, precisely, and exactly what your
params.add("c", expr="1+c1-constraint", min = 0, max = 1)
means: calculate c as 1+c1-constraint, and then apply the bounds [0, 1]. The code is doing exactly what you told it to do.
Unless you know what you are doing (I suspect maybe not ;)), I would strongly advise doing a fit with the default leastsq method before trying to use differential_evolution. It turns out that differential_evolution is not a very good global fitting method (shgo is generally better, though no "global" solver should be considered as very reliable). But, unless you know that you need such a method, you probably do not.
I would also strongly advise you to plot your data and some models evaluated with what you think are reasonable parameters.

Can I use implicit objective function in a design optimization process in Gekko. this implicit function is a subroutine that creates numerical model

I am doing an optimization for designing a seismic resistant structural system using Abaqus.
I am intending to use Gekko for this purpose. But it seems that I am making a mistakes in writing down the suitable syntax concerning this task.
"Objective" is the name of the subroutine that is responsible for creating the numerical model of Abaqus, analyzing the model, processing results and calculating the penalty function.
"Objective" returns the values of the cost of the building summed up with the penalty function to be minimized.
this is the message of error:
#error: Insufficient Data
CSV Read Error : number of data rows must be >= 2 for dynamic problems
Data Points Identified: 1
STOPPING. . .
Traceback (most recent call last):
File "C:\temp\AK\24-Gekko\opti1.10.3.0.py", line 721, in
m.solve()
File "C:\Users\amjad\AppData\Roaming\Python\Python38\site-packages\gekko\gekko.py", line 2140, in solve
raise Exception(apm_error)
Exception: #error: Insufficient Data
CSV Read Error : number of data rows must be >= 2 for dynamic problems
Data Points Identified: 1
STOPPING. . .
this is the main code of the optimization process:
'''
m = GEKKO(remote=False)
Alfa1 = m.Const(1.25)
Alfa2 = m.Const(0.3)
Alfa3 = m.Const(1.25)
UCS = m.Const(1000.) # $/Ton of steel
UCC = m.Const(67.) # $/m3 of concrete
UCCF = m.Const(20.) # $/m2 of column wood framework
UCBF = m.Const(28.) # $/m2 of beam wood framework
UCWF = m.Const(20.) # $/m2 of wall wood framework
GammaS = m.Const(7850)
GammaC = m.Const(2500)
cover = m.Const(10.)
f_c = m.Const(30.)
fy = m.Const(400.)
Est = m.Const(200000.)
eu_c = m.Const(0.003)
bw = m.Const(700.)
fai = 6.
#defining material parameters
Mats = {
"CDP30.0" :("mm",GammaC.value*1.e-12,0.18,0.,[f_c.value,0.0015,0.4,100],[f_c.value/10., 0.001 , 1.0],35,0.1,1.12,0.667,0.1,1.0,0.0),
"C30.0" :("mm",GammaC.value*1.e-12,0.18,0.,f_c.value),
"S400.0":("mm",GammaS.value*1.e-12,0.3,0.,fy.value),
}
#defining section's locations within building members and lengths of members
Sections = {
"C11":[[("A0-A1","A1-A2","D0-D1","D1-D2"),750.],[],[],[]],
"C22":[[("A2-A3","A3-A4","A4-A5","D2-D3","D3-D4","D4-D5"),750.],[],[],[]],
"C33":[[("B0-B1","C0-C1","B1-B2","C1-C2","B2-B3","C2-C3","B3-B4","C3-C4","B4-B5","C4-C5",),750.],[],[],[]],
"BB1":[[("A1-B1","C1-D1","A2-B2","C2-D2","A3-B3","C3-D3","A4-B4","C4-D4","A5-B5","C5-D5",),700.],[],[]],
"WW1":[[("B0-C1","B1-C2","B2-C3","B3-C4","B4-C5",),750.],[],[]],
}
#creating materials of the model
MatText = []
for i in Mats:
if i[1:3]=="DP":CDPs(i,Mats[i][0],Mats[i][1],Mats[i][2],Mats[i][3],Mats[i][4],Mats[i][5],Mats[i][6],Mats[i][7],Mats[i][8],Mats[i][9],Mats[i][10],Mats[i][11],Mats[i][12])
elif i[0] =="C" :EPPs(i,Mats[i][0],Mats[i][1],Mats[i][2],Mats[i][3],Mats[i][4])
elif i[0] =="S" :EPPs(i,Mats[i][0],Mats[i][1],Mats[i][2],Mats[i][3],Mats[i][4])
#objective function initializing
FX = 0.
Vars = []
for i in [x for x in Sections if x[0]=="C"]:
a = m.Var(value=125,lb=100,ub=250,integer = False)
b = m.Var(value=125,lb=100,ub=250,integer = False)
Us = m.Var(value=0.02,lb=0.01,ub=0.045,integer = False)
rebarS = rectRC(a.value,b.value,Us.value,cover.value,fai,"C")
Sections[i][1] = [a.value,b.value,Us.value,rebarS]
As = np.pi*fai**2./4*(4+2*rebarS[1][0]+2*rebarS[2][0])
#calculating the cost of a Column section
Sections[i][2] = m.Intermediate(Alfa1*As/(1000**3)*GammaS*UCS+a*b/(1000*1000)*UCC+2*(a+b)/(1000)*UCCF)
Sections[i][3] = PMinteraction(eu_c.value,cover.value,a.value,b.value,f_c.value,fy.value,Est.value,(2+rebarS[1][0])*np.pi*fai**2/4,(2+rebarS[1][0])*np.pi*fai**2/4)
FX += len(Sections[i][0][0])*Sections[i][0][1]/1000*Sections[i][2]
Vars.append((i,Sections[i][1][:3]))
for i in [x for x in Sections if x[0]=="B"]:
a = m.Var(value=125,lb=100,ub=250,integer = False)
b = m.Var(value=125,lb=100,ub=250,integer = False)
Us = m.Var(value=0.02,lb=0.01,ub=0.045)
rebarS = rectRC(a.value,b.value,Us.value,cover.value,fai,"B")
Sections[i][1] = [a.value,b.value,Us.value,rebarS]
As = np.pi*fai**2./4*(4+2*rebarS[1][0]+2*rebarS[2][0])
#calculating the cost of a Beam section
Sections[i][2] = m.Intermediate((2*Alfa2*As/(1000**3)+(1-2*Alfa2)*As/(1000**3))*GammaS*UCS+a*b/(1000*1000)*UCC+(a+2*b)/(1000)*UCBF)
print (Sections[i][2])
FX += len(Sections[i][0][0])*Sections[i][0][1]/1000*Sections[i][2]
Vars.append((i,Sections[i][1][:3]))
for i in [x for x in Sections if x[0]=="W"]:
a = m.Var(value=125,lb=100,ub=250,integer = False)
b = bw.value
Us = m.Var(value=0.009,lb=0.007,ub=0.01)
rebarS = rectRC(a.value,b.value,Us.value,cover.value,fai,"W")
Sections[i][1] = [a.value,b.value,Us.value,rebarS]
As = np.pi*fai**2./4*(4+2*rebarS[1][0]+2*rebarS[2][0])
#calculating the cost of a Wall section
Sections[i][2] = m.Intermediate(Alfa3*As/(1000**3)*GammaS*UCS+a*b/(1000*1000)*UCC+2*b/(1000)*UCWF)
print (Sections[i][2])
FX += len(Sections[i][0][0])*Sections[i][0][1]/1000*Sections[i][2]
Vars.append((i,Sections[i][1][:3]))
#modifying object function by a reference value
FX = FX/ReferenceFX * 1.
m.Minimize(Objective(Vars))
m.options.SOLVER = 1
m.options.IMODE = 6
m.solve()
'''

Gekko solution modes are described in more detail in the documentation. The current mode is IMODE=6 that should have differential and algebraic equations. In this mode, it is required to define the time points for the solution such as:
m.time=[0,0.1,0.2,0.5,1.0,1.5,2.0]
If it is a steady state solution (no differential equations) then it should be IMODE=3 for steady state optimization.
m.options.IMODE=3
There is currently no definition for PMinteraction. If this is an Abaqus model call then it will need to be replaced by a suitable model approximation that Gekko can compile into byte-code. Some options are cspline (1D), bspline (2D), or machine learning models (higher dimension functions).

How can I get sco.minimize to give me a solution that isn't the initial guesses?

I have a piece of code that worked well when I optimized advertising budget with 2 variables (channels) but when I added aditional channels, it stopped optimizing with no error messages.
import numpy as np
import scipy.optimize as sco
# setup variables
media_budget = 100000 # total media budget
media_labels = ['launchvideoviews', 'conversion', 'traffic', 'videoviews', 'reach'] # channel names
media_coefs = [0.3524764781, 5.606903166, -0.1761937775, 5.678596017, 10.50445914] #
# model coefficients
media_drs = [-1.15, 2.09, 6.7, -0.201, 1.21] # diminishing returns
const = -243.1018144
# the function for our model
def model_function(x, media_coefs, media_drs, const):
# transform variables and multiply them by coefficients to get contributions
channel_1_contrib = media_coefs[0] * x[0]**media_drs[0]
channel_2_contrib = media_coefs[1] * x[1]**media_drs[1]
channel_3_contrib = media_coefs[2] * x[2]**media_drs[2]
channel_4_contrib = media_coefs[3] * x[3]**media_drs[3]
channel_5_contrib = media_coefs[4] * x[4]**media_drs[4]
# sum contributions and add constant
y = channel_1_contrib + channel_2_contrib + channel_3_contrib + channel_4_contrib + channel_5_contrib + const
# return negative conversions for the minimize function to work
return -y
# set up guesses, constraints and bounds
num_media_vars = len(media_labels)
guesses = num_media_vars*[media_budget/num_media_vars,] # starting guesses: divide budget evenly
args = (media_coefs, media_drs, const) # pass non-optimized values into model_function
con_1 = {'type': 'eq', 'fun': lambda x: np.sum(x) - media_budget} # so we can't go over budget
constraints = (con_1)
bound = (0, media_budget) # spend for a channel can't be negative or higher than budget
bounds = tuple(bound for x in range(5))
# run the SciPy Optimizer
solution = sco.minimize(model_function, x0=guesses, args=args, method='SLSQP', constraints=constraints, bounds=bounds)
# print out the solution
print(f"Spend: ${round(float(media_budget),2)}\n")
print(f"Optimized CPA: ${round(media_budget/(-1 * solution.fun),2)}")
print("Allocation:")
for i in range(len(media_labels)):
print(f"-{media_labels[i]}: ${round(solution.x[i],2)} ({round(solution.x[i]/media_budget*100,2)}%)")
And the result is
Spend: $100000.0
Optimized CPA: $-0.0
Allocation:
-launchvideoviews: $20000.0 (20.0%)
-conversion: $20000.0 (20.0%)
-traffic: $20000.0 (20.0%)
-videoviews: $20000.0 (20.0%)
-reach: $20000.0 (20.0%)
Which is the same as the initial guesses argument.
Thank you very much!
Update: Following #joni comment, I passed the gradient function explicitly, but still no result.
I don't know how to change the constrains to test #chthonicdaemon
comment yet.
import numpy as np
import scipy.optimize as sco
# setup variables
media_budget = 100000 # total media budget
media_labels = ['launchvideoviews', 'conversion', 'traffic', 'videoviews', 'reach'] # channel names
media_coefs = [0.3524764781, 5.606903166, -0.1761937775, 5.678596017, 10.50445914] #
# model coefficients
media_drs = [-1.15, 2.09, 6.7, -0.201, 1.21] # diminishing returns
const = -243.1018144
# the function for our model
def model_function(x, media_coefs, media_drs, const):
# transform variables and multiply them by coefficients to get contributions
channel_1_contrib = media_coefs[0] * x[0]**media_drs[0]
channel_2_contrib = media_coefs[1] * x[1]**media_drs[1]
channel_3_contrib = media_coefs[2] * x[2]**media_drs[2]
channel_4_contrib = media_coefs[3] * x[3]**media_drs[3]
channel_5_contrib = media_coefs[4] * x[4]**media_drs[4]
# sum contributions and add constant (objetive function)
y = channel_1_contrib + channel_2_contrib + channel_3_contrib + channel_4_contrib + channel_5_contrib + const
# return negative conversions for the minimize function to work
return -y
# partial derivative of the objective function
def fun_der(x, media_coefs, media_drs, const):
d_chan1 = 1
d_chan2 = 1
d_chan3 = 1
d_chan4 = 1
d_chan5 = 1
return np.array([d_chan1, d_chan2, d_chan3, d_chan4, d_chan5])
# set up guesses, constraints and bounds
num_media_vars = len(media_labels)
guesses = num_media_vars*[media_budget/num_media_vars,] # starting guesses: divide budget evenly
args = (media_coefs, media_drs, const) # pass non-optimized values into model_function
con_1 = {'type': 'eq', 'fun': lambda x: np.sum(x) - media_budget} # so we can't go over budget
constraints = (con_1)
bound = (0, media_budget) # spend for a channel can't be negative or higher than budget
bounds = tuple(bound for x in range(5))
# run the SciPy Optimizer
solution = sco.minimize(model_function, x0=guesses, args=args, method='SLSQP', constraints=constraints, bounds=bounds, jac=fun_der)
# print out the solution
print(f"Spend: ${round(float(media_budget),2)}\n")
print(f"Optimized CPA: ${round(media_budget/(-1 * solution.fun),2)}")
print("Allocation:")
for i in range(len(media_labels)):
print(f"-{media_labels[i]}: ${round(solution.x[i],2)} ({round(solution.x[i]/media_budget*100,2)}%)")

The reason you are not able to solve this exact problem turns out to be all about the specific coefficients you have. For the problem as it is specified, the optimum appears to be near allocations where some spends are zero. However, at spends near zero, due to the negative coefficients in media_drs, the objective function rapidly becomes infinite. I believe this is what is causing the issues you are experiencing. I can get a solution with success = True by manipulating the 6.7 to be 0.7 in the coefficients and setting lower bound that is larger than 0 to stop the objective function from exploding. So this isn't so much of a programming issue as a problem formulation issue.
I cannot imagine it would be true that you would see more payoff when you reduce the budget on a particular item, so all the negative powers in media_dirs seem off to me.
I will also post here some improvements I made while debugging this issue. Notice that I'm using numpy arrays more to make some of the functions easier to read. Also notice how I have calculated a correct jacobian:
import numpy as np
import scipy.optimize as sco
# setup variables
media_budget = 100000 # total media budget
media_labels = ['launchvideoviews', 'conversion', 'traffic', 'videoviews', 'reach'] # channel names
media_coefs = np.array([0.3524764781, 5.606903166, -0.1761937775, 5.678596017, 10.50445914]) #
# model coefficients
media_drs = np.array([-1.15, 2.09, 1.7, -0.201, 1.21]) # diminishing returns
const = -243.1018144
# the function for our model
def model_function(x, media_coefs, media_drs, const):
# transform variables and multiply them by coefficients to get contributions
channel_contrib = media_coefs * x**media_drs
# sum contributions and add constant
y = channel_contrib.sum() + const
# return negative conversions for the minimize function to work
return -y
def model_function_jac(x, media_coefs, media_drs, const):
dy_dx = media_coefs * media_drs * x**(media_drs-1)
return -dy_dx
# set up guesses, constraints and bounds
num_media_vars = len(media_labels)
guesses = num_media_vars*[media_budget/num_media_vars,] # starting guesses: divide budget evenly
args = (media_coefs, media_drs, const) # pass non-optimized values into model_function
con_1 = {'type': 'ineq', 'fun': lambda x: media_budget - sum(x)} # so we can't go over budget
constraints = (con_1,)
bound = (10, media_budget) # spend for a channel can't be negative or higher than budget
bounds = tuple(bound for x in range(5))
# run the SciPy Optimizer
solution = sco.minimize(
model_function, x0=guesses, args=args,
method='SLSQP',
jac=model_function_jac,
constraints=constraints,
bounds=bounds
)
# print out the solution
print(solution)
print(f"Spend: ${round(float(media_budget),2)}\n")
print(f"Optimized CPA: ${round(media_budget/(-1 * solution.fun),2)}")
print("Allocation:")
for i in range(len(media_labels)):
print(f"-{media_labels[i]}: ${round(solution.x[i],2)} ({round(solution.x[i]/media_budget*100,2)}%)")
This solution at least "works" in the sense that it reports a successful solve and returns an answer different from the initial guess.

cvxpy is solving to produce empty answer

I am working with the following code:
import sys, numpy as np
import cvxpy as cvx
if __name__ == '__main__':
sims = np.random.randint(20, 30, size=500)
center = 30
n = [500, 1]
# minimize p'*log(p)
# subject to
# sum(p) = 1
# sum(p'*a) = target1
A = np.mat(np.vstack([np.ones(n[0]), sims]))
b = np.mat([1.0, center]).T
x = cvx.Variable(n)
obj = cvx.Maximize(cvx.sum(cvx.entr(x)))
constraints = [A # x == b]
prob = cvx.Problem(obj, constraints)
prob.solve()
weights = np.array(x.value)
Here the x.value is empty. I am not sure how to modify my above setup. I am trying to readjust the mean of sims to a different value defined by variable center here.

Remember to check if prob.value is finite before trying to access the values of the variables after calling prob.solve(). As you have a maximization problem, and prob.value returns -inf (see below output), it means that your problem is infeasible:
import sys, numpy as np
import cvxpy as cvx
if __name__ == '__main__':
sims = np.random.randint(20, 30, size=500)
center = 30
n = [500, 1]
# minimize p'*log(p)
# subject to
# sum(p) = 1
# sum(p'*a) = target1
A = np.mat(np.vstack([np.ones(n[0]), sims]))
b = np.mat([1.0, center]).T
x = cvx.Variable(n)
obj = cvx.Maximize(cvx.sum(cvx.entr(x)))
constraints = [A # x == b]
prob = cvx.Problem(obj, constraints)
prob.solve()
print(prob.value)
weights = np.array(x.value)
Output:
-inf
From Variable values return 'None' after solving the problem:
Diagnosing infeasibility issues is a common task when using optimization models in practice. Usually you will find either a bug in your code, or you will see that the abstract mathematical model can be infeasible (even if coded up perfectly).
For a quick reference of how it just might be how your abstract mathematical model is infeasible, rather than a bug in your code, you can try either replacing the
constraints = [A # x == b]
with
constraints = [A # x >= b] # Outputs 183.9397...
or with
constraints = [A # x <= b] # Outputs 6.2146...
and you will see that your code works.

First in way of debugging:
try and use this to see what is the issue:
prob.solve(verbose=True)
and this to check that a solution was found:
print(prob.status)
In your case the problem is infeasible, the linear problem you are trying to solve - doesn't always have a solution. You may introduce an "eps" variable to define the needed accuracy for your problem, or test in advance using a linear algebra library that some solution exists.

Pyomo.dae - Solving a system of DAEs with Casadi solver

i am trying to solve a system of DAE using pyomo.
This is a toy example
from pyomo.environ import *
from pyomo.dae import *
m = ConcreteModel()
m.r = ContinuousSet(bounds = (0., 1.))
m.t = ContinuousSet(bounds = (0., 5.))
m.c = Var(m.r, m.t)
m.dcdt = DerivativeVar(m.c, wrt = m.t)
discretizer = TransformationFactory('dae.finite_difference')
discretizer.apply_to(m, nfe=20, wrt = m.r, scheme = 'BACKWARD')
# setting initial conditions
m.c[:, 0].fix(5)
def _dae_rule(m, r, t):
return 0 == - m.c[r, t] - m.dcdt[r, t] # note that rewriting to ODE is not desired
m.ode = Constraint(m.r, m.t, rule = _dae_rule)
sim = Simulator(m, package = "casadi")
tsim, profiles = sim.simulate(numpoints=100, integrator="idas")
Unfortunately, execution leads to the error message
DAE_Error: Currently the simulator may only be applied to Pyomo models with a single ContinuousSet
How so? Only m.t is a ContinuousSet?
Manually deleting the ContinuousSet, instead using a DiscreteSet in the first place yields the error message
DAE_Error: Cannot simulate a differential equation with multiple DerivativeVars
I don't understand. Every equation only depends on its own derivative?
Also, if i were to also discretize m.t can i then use any alternative solver that might work?
Thank you very much :)

according to the documentation on Simulator, it only supports models with 1 ContinuousSet and you have m.r and m.t. Maybe you can define a system of DAEs as a function of t, at discrete values of r, or vice versa.