I want to solve two simultaneous equations using the scipy.optimize.minimize function in Python, specifically with the dog-leg trust-region algorithm. This requires me to specify the Jacobian of the problem by using scipy.optimize.approx_fprime, as suggested in one solution to my other post.
My MWE is:
import numpy as np
from scipy.integrate import quad
from scipy.optimize import minimize,approx_fprime
def myfunc(guess,a,b,c,d):
# initial guesses
x0 = guess[0]
x1 = guess[1]
# functions
z0 = lambda x: c*np.sqrt(a**3*x0)*np.sin(x)/x0**b
z1 = lambda x: np.cos(b*x0*x)/x1**a
# numerical integration with 'quad'
z0int = quad(z0,-200,200,epsabs=1e-8,epsrel=1e-6)[0] - 3.2*d
z1int = quad(z1,-200,200,epsabs=1e-8,epsrel=1e-6)[0] + c
return (z0int,z1int)
# constants
a = 0.2
b = 1.1
c = 2.5
d = 0.9
guess = np.array([0.3,0.02]) # initial guesses
myJac = approx_fprime(guess,myfunc,1e-9,a,b,c,d) # Jacobian
# minimisation, want to find x0 such that z0int=0 and z1int=0
xopt = minimize(myfunc,guess,args=(a,b,c,d),method='dogleg',jac=myJac)
print(xopt)
However I get an error TypeError: unsupported operand type(s) for -: 'tuple' and 'tuple'. I'm not really familiar with the Python optimization functions so could you please explain what is wrong and how to correct the code?
For minimisation, your function should return a single integer. You are returning a tuple, so that is the problem. The minimize function checks if the new value is lower then the old one (thus subtract) but it wants to subtract tuples instead of ints.
Change your code to only return a single integer from the function you wish to minimize
EDIT according to comments
def myfunc(guess,a,b,c,d):
# initial guesses
x0 = guess[0]
x1 = guess[1]
# functions
z0 = lambda x: c*np.sqrt(a**3*x0)/x0**b
z1 = lambda x: np.cos(b*x0)/x1**a
# numerical integration with 'quad'
z0int = quad(z0,-200,200,epsabs=1e-8,epsrel=1e-6)[0] - 3.2*d
z1int = quad(z1,-200,200,epsabs=1e-8,epsrel=1e-6)[0] + c
return (z0int,z1int) # <-- This should only return 1 single integer
For solving a system of equations, you are aiming to minimize the sum of squares of left-hand sides. For this, you might be better off using least_squares instead of more general minimize. There is an example in the documentation, https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html
Under the hood it uses a trust-region type approach.
You can rewrite the function to yield both elements, one at a time, instead of returning a tuple. One solution will be returned the 1st time, 3rd time, ..., odd times; the other solution returned every even time. You can then write a new function that you minimize instead; this new function will initialize 2 lists (evens + odds), using every other yielded element for each list. This new function will return some type of error metric with respect to both solutions such that its minimization yields the best 2 solutions.
Related
So i am doing this assignment, where i am supposed to minimize the chi squared function. I saw someone doing this on the internet so i just copied it:
Multiple variables in SciPy's optimize.minimize
I made a chi-squared function which is a function in 3 variables (x,y,sigma) where sigma is random gaussian fluctuation random.gauss(0,sigma). I did not print that code here because on first sight it might be confusing (I used a lot of recursion). But i can assure you that this function is correct.
now this code just makes a list of the calculated minimization(Which are different every time because of the random gaussian fluctuation). But here comes the main problem. If i did my calculation correctly, we should get a list with a mean of 2 (since i have 2 degrees of freedom as you can see in this link: https://en.wikipedia.org/wiki/Chi-squared_test).
def Chi2(pos):
return Chi(pos[0],pos[1],1)
x_list= []
y_list= []
chi_list = []
for i in range(1000):
result = scipy.optimize.minimize(Chi2,[5,5]).x
x_list.append(result[0])
y_list.append(result[1])
chi_list.append(Chi2(result))
But when i use this code i get a list of mean 4, however if i add the method "Powell" i get a mean of 9!!
So my main question is, how is it possible these means are so different and how do i know which method to use to get the best optimization?
Because i think the error might be in my chisquare function i will show this one as well. The story behind this assignment is that we need to find the position of a mobile device and we have routers on the positions (0,0),(20,0),(0,20) and (20,20). We used a lot of recursion, and the graph of the chi_squared looked fine(it has a minimum on (5,5)
def perfectsignal(x_m,y_m,x_r,y_r):
return 20*np.log10(c / (4 * np.pi * f)) - 10 * np.log((x_m-x_r)**2 + (y_m-y_r)**2 + 2**2)
def signal(x_m,y_m,x_r,y_r,sigma):
return perfectsignal(x_m,y_m,x_r,y_r) + random.gauss(0,sigma)
def res(x_m,y_m,x_r,y_r,sigma,sigma2):
x = (signal(x_m,y_m,x_r,y_r,sigma) - perfectsignal(x_m,y_m,x_r,y_r))/float(sigma2);
return x
def Chi(x,y,sigma):
return(res(x,y,0,0,sigma,1)**2+res(x,y,20,0,sigma,1)**2+res(x,y,0,20,sigma,1)**2+res(x,y,20,20,sigma,1)**2)
Kees
I want to integrate a function using python, where the output is a new function rather than a numerical value. For example, I have the equation (from Arnett 1982 -- analytical description of a supernova):
def A(z,tm,tni):
y=tm/(2*tni)
tm=8.8 # diffusion parameter
tni=8.77 # efolding time of Ni56
return 2*z*np.exp((-2*z*y)+(z**2))
I want to then find the integral of A, and then plot the results. First, I naively tried scipy.quad:
def Arnett(t,z,tm,tni,tco,Mni,Eni,Eco):
x=t/tm
Eni=3.90e+10 # Heating from Ni56 decay
Eco=6.78e+09 # Heating from Co56 decay
tni=8.77 # efolding time of Ni56
tco=111.3 # efolding time of Co56
tm=8.8 # diffusion parameter
f=integrate.quad(A(z,tm,tni),0,x) #integral of A
h=integrate.quad(B(z,tm,tni,tco),0,x) #integral of B
g=np.exp((-(x/tm)**2))
return Mni*g*((Eni-Eco)*f+Eco*h)
Where B is also a pre-defined function (not presented here). Both A and B are functions of z, however the final equation is a function of time, t. (I believe that it is herein I am causing my code to fail.)
The integrals of A and B run from zero to x, where x is a function of time t. Attempting to run the code as it stands gives me an error: "ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()".
So after a short search I thought that maybe sympy would be the way to go. However I am failing with this as well.
I wonder if anyone has a helpful suggestion how to complete this task please?
Many thanks,
Zach
You can integrate A analytically. Assuming I'm not missing something silly due to being up way too late, does the following help?
import sympy as sy
sys.displayhook = sy.pprint
A, y, z, tm, t, tni = sy.symbols('A, y, z, tm, t, tni')
A = 2*z*sy.exp(-2*z*y + z**2)
expr = sy.integrate(A, (z,0,t)) # patience - this takes a while
expr
# check:
(sy.diff(expr,t).simplify() - A.replace(z,t)).simplify()
# thus, the result:
expr.replace(y,tm/(2*tni)).replace(t,t/tm)
The last line yields the integral of your A function in analytic form, though it does require evaluating the imaginary error function (which you can do with scipy.special.erfi()).
I think what you are looking for are lambda expression (if i understood correctly what you said.. see here for extra information and some examples on lambda functions).
What they allow you to do is define an anonymous function in A and return it so that you get your B function, should work something like this:
def A(parameters):
return lambda x: x * parameters # for simplicity i applied a multiplication
# but you can apply anything you want to x
B = A(args)
x = B(2)
Hope I could provide you with a decent response!
I think the error you get comes from an incorrect call to scipy.integrate.quad:
The first argument needs to be just the function name, integration is then performed over the first variable of this function. The values of the other variables can be passed to the function via the args keyword.
The output of scipy.integrate.quad contains not only the value of the integral, but also an error estimate. So a tuple of 2 values is returned!
In the end the following function should work:
def Arnett(t, z, Mni, tm=8.8, tni=8.77, tco=111.3, Eni=3.90e+10,
Eco=6.78e+09):
x=t/tm
f,err=integrate.quad(A,0,x,args=(tm,tni)) #integral of A
h,err=integrate.quad(B,0,x,args=(tm,tni,tco)) #integral of B
g=np.exp((-(x/tm)**2))
return Mni*g*((Eni-Eco)*f+Eco*h)
But an even better solution would probably be integrating A and B analytically and then evaluating the expression as murison suggested.
Why does the following code return a ValueError?
from scipy.optimize import fsolve
import numpy as np
def f(p,a=0):
x,y = p
return (np.dot(x,y)-a,np.outer(x,y)-np.ones((3,3)),x+y-np.array([1,2,3]))
x,y = fsolve(f,(np.ones(3),np.ones(3)),9)
ValueError: setting an array element with a sequence.
The basic problem here is that your function f does not satisfy the criteria required for fsolve to work. These criteria are described in the documentation - although arguably not very clearly.
The particular things that you need to be aware of are:
the input to the function that will be solved for must be an n-dimensional vector (referred to in the docs as ndarray), such that the value of x you want is the solution to f(x, *args) = 0.
the output of f must be the same shape as the x input to f.
Currently, your function takes a 2 member tuple of 1x3-arrays (in p) and a fixed scalar offset (in a). It returns a 3 member tuple of types (scalar,3x3 array, 1x3 array)
As you can see, neither condition 1 nor 2 is met.
It is hard to advise you on exactly how to fix this without being exactly sure of the equation you are trying to solve. It seems you are trying to solve some particular equation f(x,y,a) = 0 for x and y with x0 = (1,1,1) and y0 = (1,1,1) and a = 9 as a fixed value. You might be able to do this by passing in x and y concatenated (e.g. pass in p0 = (1,1,1,1,1,1) and in the function use x=p[:3] and y = p[3:] but then you must modify your function to output x and y concatenated into a 6-dimensional vector similarly. This depends on the exact function your are solving for and I can't work this out from the output of your existing f (i.e based on a dot product, outer product and sum based tuple).
Note that arguments that you don't pass in the vector (e.g. a in your case) will be treated as fixed values and won't be varied as part of the optimisation or returned as part of any solution.
Note for those who like the full story...
As the docs say:
fsolve is a wrapper around MINPACK’s hybrd and hybrj algorithms.
If we look at the MINPACK hybrd documentation, the conditions for the input and output vectors are more clearly stated. See the relevant bits below (I've cut some stuff out for clarity - indicated with ... - and added the comment to show that the input and output must be the same shape - indicated with <--)
1 Purpose.
The purpose of HYBRD is to find a zero of a system of N non-
linear functions in N variables by a modification of the Powell
hybrid method. The user must provide a subroutine which calcu-
lates the functions. The Jacobian is then calculated by a for-
ward-difference approximation.
2 Subroutine and type statements.
SUBROUTINE HYBRD(FCN,N,X, ...
...
FCN is the name of the user-supplied subroutine which calculates
the functions. FCN must be declared in an EXTERNAL statement
in the user calling program, and should be written as follows.
SUBROUTINE FCN(N,X,FVEC,IFLAG)
INTEGER N,IFLAG
DOUBLE PRECISION X(N),FVEC(N) <-- input X is an array length N, so is output FVEC
----------
CALCULATE THE FUNCTIONS AT X AND
RETURN THIS VECTOR IN FVEC.
----------
RETURN
END
N is a positive integer input variable set to the number of
functions and variables.
X is an array of length N. On input X must contain an initial
estimate of the solution vector. On output X contains the
final estimate of the solution vector.
I am looking for the correct approach to use a variable number of parameters as input for the optimizer in scipy.
I have a set of input parameters p1,...,pn and I calculate a quality criteria with a function func(p1,...,pn). I want to minimize this value.
The input parameters are either 0 or 1 indicating they should be used or not. I cannot simply delete all unused ones from the parameter list, since my function for the quality criteria requires them to be "0" to remove unused terms from equations.
def func(parameters):
...calculate one scalar as quality criteria...
solution = optimize.fmin_l_bfgs_b(func,parameters,approx_grad=1,bounds=((0.0, 5.0),...,(0.0,5.0)) # This will vary all parameters
Within my code the optimizer runs without errors, but of course all given parameters are changed to achieve the best solution.
Is there a way to have e.g. 10 input parameters for func, but only 5 of them are used in the optimizer?
So far I can only think of changing my func definition in a way that I will not need the "0" input from unused parameters. I would appreciate any ideas how to avoid that.
Thanks a lot for the help!
If I understand correctly, you are asking for a constrained best fit, such that rather than finding the best [p0,p1,p2...p10] for function func(), you want to find the best best [p0, p1, ...p5] for function func() under a condition that p6=fixed6, p7=fixed7, p8=fixed8... and so on.
Translate it into python code is straight forward if you use args=(somthing) in scipy.optimize.fmin_l_bfgs_b. Firstly, write a partially fixed function func_fixed()
def func_fixed(p_var, p_fixed):
return func(p_var+p_fixed)
# this will only work if both of them are lists. If they are numpy arrays, use hstack, append or similar
solution = optimize.fmin_l_bfgs_b(func_fixed,x0=guess_parameters,\
approx_grad=your_grad,\
bounds=your_bounds,\
args=(your_fixed_parameters), \ #this is the deal
other_things)
It is not necessary to have func_fixed(), you can use lambda. But it reads much easier this way.
I recently solved a similar problem where I want to optimise a different subset of parameters at each run but need all parameters to calculate the objective function. I added two arguments to my objective function:
an index array x_idx which indicates which parameters to optimise, i.e. 0 don't optimise and 1 optimise
an array x0 with the initial values of all parameters
In the objective function I set the list of the parameters according to the index array either to the parameters which are to be optimised or the initial values.
import numpy
import scipy.optimize
def objective_function(x_optimised, x_idx, x0):
x = []
j = 0
for i, idx in enumerate(x_idx):
if idx is 1:
x.append(x_optimised[j])
j = j + 1
else:
x.append(x0[i])
x = numpy.array(x)
return sum(x**2)
if __name__ == '__main__':
x_idx = [1, 1, 0]
x0 = [1.1, 1.3, 1.5]
x_initial = [x for i, x in enumerate(x0) if x_idx[i] is 1]
xopt, fopt, iter, funcalls, warnflag = scipy.optimize.fmin(objective_function, \
x_initial, args=(x_idx, x0,), \
maxfun = 200, full_output=True)
print xopt
I am trying to optimize (minimize) a two dimensional function E(n,k) defined as follows:
error=lambda x,y,w: (math.log(abs(Tformulated(x,y,w))) - math.log(abs(Tw[w])))**2 + (math.atan2(Tformulated(x,y,w).imag,Tformulated(x,y,w).real) - math.atan2(Tw[w].imag,Tw[w].real))**2
where Tformulated is obtained as follows :
def Tformulated(n,k,w):
z=1j
L=1
C=0.1
RC=(w*L)/C
n1=complex(1,0)
n3=complex(1,0)
n2=complex(n,k)
FP=1/(1-(((n2-n1)/(n2+n1))*((n2-n3)/(n2+n3))*math.exp(-2*z*n2*RC)))
Tform=((2*n2*(n1+n3))/((n2+n1)*(n2+n3)))*(math.exp(-z*(n2-n1)*RC))*FP
return Tform
and Tw is a list previously calculated having complex valued elements.
What I am exactly trying to do is for each value of w (used in "error x,y,w ....") I want to minimize the function "error" for the values of x & y. w ranges from 1 to 2048. So, it is basically a 2D minimization problem. I have tried programming on my part (though I am getting stuck at what method to use and how to use it); my code is as follows :
temp=[]
i=range(5)
retval = fmin_powell(error , x ,y, args=(i) , maxiter=100 ,maxfun=100)
temp.append(retval)
I am not sure even if fmin_powell is the correct way to go.
Here's a simplest example:
from scipy.optimize import fmin
def minf(x):
return x[0]**2 + (x[1]-1.)**2
print fmin(minf,[1,2])
[out]:
Optimization terminated successfully.
Current function value: 0.000000
Iterations: 44
Function evaluations: 82
[ -1.61979362e-05 9.99980073e-01]
A possible gotcha here is that the minimization routines are expecting a list as an argument. See the docs for all the gory details. Not sure if you can minimize complex-valued functions directly, you might need to consider the real and imaginary parts separately.