I'm currently trying to solve numerically a minimization problem and I tried to use the optimization library available in SciPy.
My function and derivative are a bit too complicated to be presented here, but they are based on the following functions, the minimization of which do not work either:
def func(x):
return np.log(1 + np.abs(x))
def grad(x):
return np.sign(x) / (1.0 + np.abs(x))
When calling the fmin_bfgs function (and initializing the descent method to x=10), I get the following message:
Warning: Desired error not necessarily achieved due to precision loss.
Current function value: 2.397895
Iterations: 0
Function evaluations: 24
Gradient evaluations: 22
and the output is equal to 10 (i.e. initial point). I suppose that this error may be caused by two problems:
The objective function is not convex: however I checked with other non-convex functions and the method gave me the right result.
The objective function is "very flat" when far from the minimum because of the log.
Are my suppositions true? Or does the problem come from anything else?
Whatever the error can be, what can I do to correct this? In particular, is there any other available minimization method that I could use?
Thanks in advance.
abs(x) is always somewhat dangerous as it is non-differentiable. Most solvers expect problems to be smooth. Note that we can drop the log from your objective function and then drop the 1, so we are left with minimizing abs(x). Often this can be done better by the following.
Instead of min abs(x) use
min t
-t <= x <= t
Of course this requires a solver that can solve (linearly) constrained NLPs.
Related
I am looking into using Nlopt for solving optimisation problems in Python.
I have a series of simultaneous equations of the form
Ax = b
where A is an NxM matrix, with x the solution. Another way to think about this is that I have N simultaneous equations of the form x_1c_1m + x_2c_2m + .... + x_Nc_Nm = k_M, where x_i are variables to solve for, c_im is a constant associated with x_i when in equation M=m, and k_m is some constant in equation M=m. c_im and k_m are all known.
What confuses me is how to even approach this in Nlopt. Nlopt requires you to have actual callable functions, which I don't have? I suppose I could generalise each of the equations in that matrix equation above to something like:
def fn(x,c_m,k_m):
val = 0
for x_i, c_im in zip(x,c_m):
val += x_i * c_im
return val - k_m
where c_m and k_m would be already known, with the variables to solve for in x. All the examples I've seen have only been looking at a single variable problem, which has kind of thrown me a little. Would I then have to somehow define M copies of this function, and set each copy of fn as an equality constraint in the Nlopt optimisation object? It's all rather confusing. I'm looking to solve for x, which itself has multiple solutions, and I want to try to find the minimum values of x (or atleast an approximate solution if an exact solution cannot be found). Would I have to then set multiple objective functions, ie obj_fn_i = min(x_i) or something like that? It's all a little confusing to me in terms of what needs to be presented to the solver. I've already got an analytical solution to the above problem, so I can check my results reliably. Any help appreciated.
Cheers!
I have been using NLopt for a couple of problems, and what I have come to understand is the solver requires an objective function which returns a float value, so you must set the function as an MSE sum, or still as a single float value to be minimized. And it can solve for an array of variables x, in which both the objective function and constraint must depend. All equations that are involved in the system you can insert either in the objective function directly, or as constraints.
Hope this was helpful somehow!
I'm trying to compute the triple integral of the function abs(x-y)**(H-1)*abs(y-z)**(H-1)*abs(z-x)**(H-1)
on [0,1]^3 for example with a H between (0.5,1), However it seems hard for python to compute it.
So, firstly, i tried with integrate.tplquad from scipy but it's enable to do it, it only returns that the integral is probably divergent or slowly convergent.
To avoid this,I recode a Riemann function by the well-known method and I took an "epsilon" which is a small postive number such as 10**-6 in the function itself in every absolute value, but it's also necessary to take another epsilon on the interval
The first error, i noticed is that 0.0 cannot be raised to a negative power. But then doing that transformation, knowing the answer should be around 29.7, Python returns incompatible value.
I think the problem deals with numeric issue, or the integration scheme itself, although my function Riemann isn't otptimized, i think it should have been close to the real value I expected.
Here's the code
def f(H):
eps=10**-12
return lambda x,y,z:(abs(x-y)+eps)**(H-1)*abs(y-z+eps)**(H-1)*abs(z-x+eps)**(H-1)
def riemann(H,g,a,b,c,d,e,h,n):
s=0
du=(b-a)/n
dv=(c-d)/n
dw=(e-h)/n
for i in range (n):
for j in range(n):
for k in range(n):
s+=g(du*i,dv*j,dw*k)
s=s/(n**3)
return s
Thanks for your help.
So far I understand the minimize function with method Trust-ncg, the "method specific" parameter "max_trust_radius" is the maximum value for a new step optimization.
However, I experience a weird behaviour.
I work in my doctorate data and I have a code that invokes minimize function (with trust ncg method)
passing parameters
{
'initial_trust_radius':0.1,
'max_trust_radius':1,
'eta':0.15,
'gtol':1e-5,
'disp': True
}
I invoke minimize function as:
res = minimize(bbox, x0, method='trust-ncg',jac=bbox_der, hess=bbox_hess,options=opt_par)
where
bbox is a function to evaluate the objective function
x0 is the initial guess
bbox_der is the gradient function
bbox_hess hessian function
opt_par is the dictionary above with the parameters.
Bbox invokes simulation code and get the data. It works: minimize go back and forth, proposing new values, bbox invokes simulation.
Everything works well until I got a weird issue.
The "x" vector contains 8 values. I realize that one of the iterations, the last value is greater than 1.
Per the max_trust_radius, I think that it should be less than 1, but it is 1.0621612802208713e+00
The issue causes problems because bbox can not receive the value greater than 1, as it invokes a simulation program and there is a constraint that it can not receive 1 or greater than 1.
I found the scipy code and tried to see if I could be able to find a bug or something wrong but I am not.
My main concerns are:
My understanding is that there is a bug in the scipy minimize code as the new value is greater than max_trust_radius .
How can I manipulate or control the values to avoid that values became greater than 1?
Do you suggest something to investigate the issue?
The max_trust_radius controls how large steps you are allowed to take:
max_trust_radius : float
Maximum value of the trust-region radius.
No steps that are longer than this value will be proposed.
Since you are very likely to take many steps during the minimization, each which can be up to 1 long, it is not strange at all that you (assuming ||x0||=0) end up with ||x|| > 1.
If your problem is strictly bounded then you need to apply an optimization algorithm that supports bounds on the parameters.
For scipy.optimize.minimize only L-BFGS-B, TNC and SLSQP methods seem to support the bounds= keyword.
I'm having trouble optimizing a very simple function I'm using as a test case before moving on to something more complex. I've tried different optimization methods, giving the method a bound and even giving the exact solution as the initial guess.
Function I'm trying to optimize: f(x) = 1 / x - x
Here is my code:
import scipy
def testfun(x): return (1 / x - x)
sol = scipy.optimize.minimize(testfun, 1).x
it returns large numbers (3.2 e+08) as the solution
Am I using the optimization function incorrectly?
As Victor mentioned, the optimization function is working correctly,
I was looking to solve f(x) --> 0 which requires a root finding method rather than an optimization routine.
for example:
scipy.optimize.root(testfun, 1) or scipy.optimize.Newton(testfun, 1)
I try with fmin_bfgs to find the local minimum of the absolute function abs(x). The initial point is set to 100.0; the expected answer is 0.0. However, I get:
In [184]: op.fmin_bfgs(lambda x:np.abs(x),100.0)
Warning: Desired error not necessarily achieved due to precision loss.
Current function value: 100.000000
Iterations: 0
Function evaluations: 64
Gradient evaluations: 20
Out[184]: array([100.0])
Why?
Methods like fmin_bfgs and fmin_slsqp require smooth (continuous derivative) functions in order to provide reliable results. abs(x) has a dicontinuous derivative at its minimum. A method like the Nelder-Mead simplex, which doesn't require continuous derivatives, might provide better results in this case.