I have a question regarding solving a minimization problem using scipy.optimize in python. I have an 1-D array (x) containing about 2000 elements as the variables of this problem, and a list of {constraint,dict}elements as the constraints of the optimization problem.The issue is that I want to calculate the sum of a large number of variables within a for loop to define objective function and the constraints. I have attached a simple example of my code below, but it should be considered that according to the complication of my code, it is not possible to do the calculation without for loops. However, by doing this, I face with this error: 'function' object is not iterable.
def objective(x):
sum = 0
for i in range(1, 1000):
sum += x[i]
return sum
def constraints(x):
constraint = []
for i in range(1, 1000):
sum = 0
for j in range(1, 100):
sum += j*x[i]
constraint.append({'type': 'ineq', 'fun': 10 - sum})
return constraint
sol = minimize(objective, x0, method='slsqp', constraints=constraints)
looked at the video you referenced, think you may have misunderstood a couple of things. I'll start with just a basic example to get you started.
If we take the below code:
from scipy.optimize import minimize
def objective(x):
x1=x[0]
x2=x[1]
x3=x[2]
x4=x[3]
return x1+x2+x3+x4
x0=(1,1,1,1)
solution=minimize(objective,x0)
Simple setup. You see the python function objective is returning a math function. That math function is what you are minimizing. Note how all your variables are defined by calling the value of a list of values (x0 in this case). Thus x1 is x0[0], x2 is x0[1], etc.
In your above example, you don't define x0, and you don't have an equation. Finally, sum is a python function (it takes the sum of a list). Instead, to do what you wish to do, lets rewrite the above using a for loop now:
from scipy.optimize import minimize
import numpy as np
def objective(x):
equation = 0
for i in range(4):
equation += x[i]
return equation
x0=np.ones(4)
solution=minimize(objective,x0)
This will give you the exact same output as above; however now you can see I've changed sum to equation (now you won't get issues with built in python functions), and you've now defined your variables). I'm not going to go through all your code, but I hope this example clears enough up that it gets you going.
If you want 1000s values of x, just give x0 an array of 1000 values.
Edit:
To resolve the issue in the 2nd part of your code:
Presuming you fixed using sum. Again, constraints is a dict where you input a function (mathematical). I don't get the error that you are getting, so I don't know where that is coming from. But you can create a constraint function using your above example.
def constraints(x):
constraint = []
for i in range(4):
value = 0
for j in range(4):
value += 10-j*x[i]
constraint.append(value)
return constraint
x0=np.ones(4)
con={'type': 'ineq', 'fun': constraints}
sol = minimize(objective, x0, method='slsqp', constraints=con)
Edit 2:
You can use the exact same ideology above to create a 2nd constraint:
from scipy.optimize import minimize
import numpy as np
def objective(x):
equation = 0
for i in range(4):
equation += x[i]
return equation
def constraints(x):
constraint = []
for i in range(4):
value = 0
for j in range(4):
value += 10-j*x[i]
constraint.append(value)
return constraint
def constraints2(x):
constraint2 = []
for a in range(4):
value = 0
for b in range(4):
value += 5-a*x[b]
constraint2.append(value)
return constraint2
x0=np.ones(4)
con={'type': 'ineq', 'fun': constraints}
con2={'type': 'eq', 'fun': constraints2}
combined=[con,con2]
sol = minimize(objective, x0, method='slsqp', constraints=combined)
This works without any problems or errors, so again, I don't see the problem you are having.
Related
I usually rely on Wolfram Mathematica for this kind of thing, but I've been delving into Python recently and the results are so much better. Basically, I'm looking for numerical solutions for systems like the following.
system:
Well, I know that there are solutions, because Wolfram Mathematica found a single one (0.0858875,0.0116077,-0.156661,1.15917). What I tried to do in Python is this brute force code.
import numpy as np
START = -3
END = 3
STEP = 0.1
for r0 in np.arange(START, END, STEP):
for r1 in np.arange(START, END, STEP):
for r2 in np.arange(START, END, STEP):
for r3 in np.arange(START, END, STEP):
eq0 = r0*r2+r1*r3
eq1 = r0*r1+r1*r2+r2*r3+r0*r3
eq2 = r0**2+r1**2+r2**2+r3**2-4*(r0+r1+r2+r3)**2
if (eq0 == 0 and eq1 < 0 and eq2 < 0):
print(r0, r1, r2, r3)
Edit: I'm okay with things like -0.00001< eq0 < 0.00001 instead of eq1 == 0
Well, although it didn't find solutions in this case, the brute force method went well for other systems I'm dealing with, particularly when there are fewer equations and variables. Starting with four variables, it becomes really difficult.
I'm sorry if I'm asking too much. I'm completely new to Python, so I also don't know if this is actually trivial. Maybe fsolve would be useful? I'm not sure if it works with inequalities. Also, even when the systems I encounter have only equalities, they always have more variables than equations, like this one:
system2:
.
Hence 'fsolve' is not appropriate, right?
As soon as your system contains inequalities, you need to formulate it as an optimization problem and solve it with scipy.optimize.minimize. Otherwise, you can use scipy.optimize.root or scipy.optimize.fsolve to solve an equation system. Note that the former is also exactly what is done behind the scenes in root and fsolve, i.e. both solve a least-squares optimization problem.
In general, the problem
g_1(x) = 0, ..., g_m(x) = 0
h_1(x) < 0, ..., h_p(x) < 0
can be formulated as
min g_1(x)**2 + ... + g_m(x)**2
s.t. -1.0*(h_1(x) + eps) >= 0
.
.
-1.0*(h_p(x) + eps) >= 0
where eps is a tolerance to model the strict inequality.
Hence, you can solve your first problem as follows:
import numpy as np
from scipy.optimize import minimize
def obj(r):
return (r[0]*r[2]+r[1]*r[3])**2
eps = 1.0e-6
constrs = [
{'type': 'ineq', 'fun': lambda r: -1.0*(r[0]*r[1] + r[1]*r[2] + r[2]*r[3] + r[0]*r[3] + eps)},
{'type': 'ineq', 'fun': lambda r: -1.0*(np.sum(r**2) - 4*(np.sum(r))**2 + eps)}
]
# res.x contains the solution
res = minimize(obj, x0=np.ones(4), constraints=constrs)
Your second problem can be solved similarly. Here, you only need to remove the constraints. Alternatively, you can use root where it's worth mentioning that it solves F(x) = 0 for a function F: R^N -> R^N, i.e. a function of N variables that returns an N dimensional vector. In case your function has fewer equations than variables, you can simply fill up the vector with zeros:
import numpy as np
from scipy.optimize import root
def F(r):
vals = np.zeros(r.size)
vals[0] = np.dot(r[:5], r[1:]) + r[0]*r[5]
vals[1] = r[0]*r[3] + r[1]*r[4] + r[2]*r[5]
vals[2] = np.sum(r**2) - 3*np.sum(r)**2
return vals
# res.x contains your solution
res = root(F, x0=np.ones(6))
Not really an answer, but you can simplify this a lot using product
from itertools import product
import numpy as np
START = -3
END = 3
STEP = 0.1
for r0, r1, r2, r3 in product(np.arange(START, END, STEP), repeat=4):
print(r0, r1, r2, r3)
Not sure if your problem is a root finding problem or a minimization with constraints problem, but take a look at scipy.optimize and scipy.linprog, maybe one of those methods can be bent to your application.
I am trying to solve for the following equation, however, there is very little documentation on the SciPy reference guide so I am not sure how to go about doing it.
I have an array of 10 uniform random variables, let's call it X.
I have the following function which takes a parameter theta and a uniform RV from the X's and returns their product raised to the power of e:
def f(x_i, theta):
return math.exp(x * theta)
I am trying to find theta such that this equation holds:
ok you can move the 20 to the other side of the equation so that it is 0, but still I am not sure how to optimize for this quantity given the summation?
Any help would be appreciated! Thank you.
I am wondering what the hiccup is. In essence, you just have a function g(theta) with
g(theta) = sum(i, f(x[i],theta))/10 - 20
I assume x is data.
So, we could do:
from scipy.optimize import root_scalar
from random import uniform, seed
from math import exp
def f(x_i, theta):
return exp(x_i * theta)
def g(theta, x):
s = 0
for x_i in x:
s += f(x_i,theta)
return s/10 - 20
def solve():
n = 10
seed(100)
x = [uniform(0,1) for i in range(n)]
print(x)
res = root_scalar(g,args=x,bracket=(0,100))
print(res)
solve()
I see:
[0.1456692551041303, 0.45492700451402135, 0.7707838056590222, 0.705513226934028, 0.7319589730332557, 0.43351443489540376, 0.8000204571334277, 0.5329014146425713, 0.08015370917850195, 0.45594588118356716]
converged: True
flag: 'converged'
function_calls: 16
iterations: 15
root: 4.856897057972411
I need an ODE-solver for a stiff problem similar to MATLAB ode15s.
For my problem I need to check how many steps (calculations) is needed for different initial values and compare this to my own ODE-solver.
I tried using
solver = scipy.integrate.ode(f)
solver.set_integrator('vode', method='bdf', order=15, nsteps=3000)
solver.set_initial_value(u0, t0)
And then integrating with:
i = 0
while solver.successful() and solver.t<tf:
solver.integrate(tf, step=True)
i += 1
print(i)
Where tf is the end of my time interval.
The function used is defined as:
def func(self, t, u):
u1 = u[1]
u2 = mu * (1-numpy.dot(u[0], u[0]))*u[1] - u[0]
return numpy.array([u1, u2])
Which with the initial value u0 = [ 2, 0] is a stiff problem.
This means that the number of steps should not depend on my constant mu.
But it does.
I think the odeint-method can solve this as a stiff problem - but then I have to send in the whole t-vector and therefore need to set the amount of steps that is done and this ruins the point of my assignment.
Is there anyway to use odeint with adaptive stepsize between two t0 and tf?
Or can you see anything I miss in the use of the vode-integrator?
I'm seeing something similar; with the 'vode' solver, changing methods between 'adams' and 'bdf' doesn't change the number of steps by very much. (By the way, there is no point in using order=15; the maximum order of the 'bdf' method of the 'vode' solver is 5 (and the maximum order of the 'adams' solver is 12). If you leave the argument out, it should use the maximum by default.)
odeint is a wrapper of LSODA. ode also provides a wrapper of LSODA:
change 'vode' to 'lsoda'. Unfortunately the 'lsoda' solver ignores
the step=True argument of the integrate method.
The 'lsoda' solver does much better than 'vode' with method='bdf'.
You can get an upper bound on
the number of steps that were used by initializing tvals = [],
and in func, do tvals.append(t). When the solver completes, set
tvals = np.unique(tvals). The length of tvals tells you the
number of time values at which your function was evaluated.
This is not exactly what you want, but it does show a huge difference
between using the 'lsoda' solver and the 'vode' solver with
method 'bdf'. The number of steps used by the 'lsoda' solver is
on the same order as you quoted for matlab in your comment. (I used mu=10000, tf = 10.)
Update: It turns out that, at least for a stiff problem, it make a huge difference for the 'vode' solver if you provide a function to compute the Jacobian matrix.
The script below runs the 'vode' solver with both methods, and it
runs the 'lsoda' solver. In each case, it runs the solver with and without the Jacobian function. Here's the output it generates:
vode adams jac=None len(tvals) = 517992
vode adams jac=jac len(tvals) = 195
vode bdf jac=None len(tvals) = 516284
vode bdf jac=jac len(tvals) = 55
lsoda jac=None len(tvals) = 49
lsoda jac=jac len(tvals) = 49
The script:
from __future__ import print_function
import numpy as np
from scipy.integrate import ode
def func(t, u, mu):
tvals.append(t)
u1 = u[1]
u2 = mu*(1 - u[0]*u[0])*u[1] - u[0]
return np.array([u1, u2])
def jac(t, u, mu):
j = np.empty((2, 2))
j[0, 0] = 0.0
j[0, 1] = 1.0
j[1, 0] = -mu*2*u[0]*u[1] - 1
j[1, 1] = mu*(1 - u[0]*u[0])
return j
mu = 10000.0
u0 = [2, 0]
t0 = 0.0
tf = 10
for name, kwargs in [('vode', dict(method='adams')),
('vode', dict(method='bdf')),
('lsoda', {})]:
for j in [None, jac]:
solver = ode(func, jac=j)
solver.set_integrator(name, atol=1e-8, rtol=1e-6, **kwargs)
solver.set_f_params(mu)
solver.set_jac_params(mu)
solver.set_initial_value(u0, t0)
tvals = []
i = 0
while solver.successful() and solver.t < tf:
solver.integrate(tf, step=True)
i += 1
print("%-6s %-8s jac=%-5s " %
(name, kwargs.get('method', ''), j.func_name if j else None),
end='')
tvals = np.unique(tvals)
print("len(tvals) =", len(tvals))
I have two programs, one that can take in N coupled ODEs and one that uses 2 coupled ODEs. In the case that I input the 2 same ODEs into both codes, with the same time span, I get different answers. I know the correct answer, so I can deduce that my N many program is wrong.
Here is the code for the 2 equation dedicated one:
# solve the coupled system dy/dt = f(y, t)
def f(y, t):
"""Returns the collections of first-order
coupled differential equations"""
#v11i = y[0]
#v22i = y[1]
#v12i = y[2]
print y[0]
# the model equations
f0 = dHs(tRel,vij)[0].subs(v12,y[2])
f1 = dHs(tRel,vij)[3].subs(v12,y[2])
f2 = dHs(tRel,vij)[1].expand().subs([(v11,y[0]),(v22,y[1]),(v12,y[2])])
return [f0, f1, f2]
# Initial conditions for graphing
v110 = 6
v220 = 6
v120 = 4
y0 = [v110, v220, v120] # initial condition vector
sMesh = np.linspace(0, 1, 10e3) # time grid
# Solve the DE's
soln = odeint(f, y0, sMesh)
and here is the N equation dedicated one:
def f(y, t):
"""Returns the derivative of H_s with initial
values plugged in"""
# the model equations
print y[0]
for i in range (0,len(dh)):
for j in range (0,len(y)):
dh[i] = dh[i].subs(v[j],y[j])
dhArray = []
for i in range(0,len(dh)):
dhArray.append(dh[i])
return dhArray
sMesh = np.linspace(0, 1, 10e3) # time grid
dh = dHsFunction(t, V_s).expand()
soln = odeint(f, v0, sMesh)
where dHs(tRel,vij) = dHsFunction(t,V_s) i.e. the exact same ODEs. Similarly y0 and v0 are the exact same. But when I print y[0] in the N many case, I get an output of:
6.0
5.99999765602
5.99999531204
5.97655553477
5.95311575749
5.92967598021
5.69527820744
5.46088043467
5.2264826619
2.88250493418
0.53852720647
-1.80545052124
-25.2452277984
-48.6850050755
-72.1247823527
-306.522555124
as opposed to the 2 dedicated case of:
6.0
5.99999765602
5.99999765602
5.99999531205
5.99999531205
5.98848712729
5.98848712125
5.97702879748
5.97702878476
5.96562028875
5.96562027486
5.91961750442
5.91961733611
5.93039037809
5.93039029335
5.89564277275
5.89564273736
5.86137647436
5.86137638807
5.82758984835
etc.
where the second result is the correct one and graphs the proper graphs.
Please let me know if more code is needed or anything else. Thanks.
Your second version for f modifies the value of the global variable dh.
On the first call, you substitute in values in it, and these same values are then used in all subsequent calls.
Avoid that by using e.g. dh_tmp = list(dh) inside the function.
I have a second order differential equation that I want to solve it in python. The problem is that for one of the variables I don't have the initial condition in 0 but only the value at infinity. Can one tell me what parameters I should provide for scipy.integrate.odeint ? Can it be solved?
Equation:
Theta needs to be found in terms of time. Its first derivative is equal to zero at t=0. theta is not known at t=0 but it goes to zero at sufficiently large time. all the rest is known. As an approximate I can be set to zero, thus removing the second order derivative which should make the problem easier.
This is far from being a full answer, but is posted here on the OP's request.
The method I described in the comment is what is known as a shooting method, that allows converting a boundary value problem into an initial value problem. For convenience, I am going to rename your function theta as y. To solve your equation numerically, you would first turn it into a first order system, using two auxiliary function, z1 = y and z2 = y', and so your current equation
I y'' + g y' + k y = f(y, t)
would be rewitten as the system
z1' = z2
z2' = f(z1, t) - g z2 - k z1
and your boundary conditions are
z1(inf) = 0
z2(0) = 0
So first we set up the function to compute the derivative of your new vectorial function:
def deriv(z, t) :
return np.array([z[1],
f(z[0], t) - g * z[1] - k * z[0]])
If we had a condition z1[0] = a we could solve this numerically between t = 0 and t = 1000, and get the value of y at the last time as something like
def y_at_inf(a) :
return scipy.integrate.odeint(deriv, np.array([a, 0]),
np.linspace(0, 1000, 10000))[0][-1, 0]
So now all we need to know is what value of a makes y = 0 at t = 1000, our poor man's infinity, with
a = scipy.optimize.root(y_at_inf, [1])