I am using a scipy.minimize function, where I'd like to have one parameter only searching for options with two decimals.
def cost(parameters,input,target):
from sklearn.metrics import mean_squared_error
output = self.model(parameters = parameters,input = input)
cost = mean_squared_error(target.flatten(), output.flatten())
return cost
parameters = [1, 1] # initial parameters
res = minimize(fun=cost, x0=parameters,args=(input,target)
model_parameters = res.x
Here self.model is a function that performs some matrix manipulation based on the parameters. Input and target are two matrices. The function works the way I want to, except I would like to have parameter[1] to have a constraint. Ideally I'd just like to give an numpy array, like np.arange(0,10,0.01). Is this possible?
In general this is very hard to do as smoothness is one of the core-assumptions of those optimizers.
Problems where some variables are discrete and some are not are hard and usually tackled either by mixed-integer optimization (working good for MI-linear-programming, quite okay for MI-convex-programming although there are less good solvers) or global-optimization (usually derivative-free).
Depending on your task-details, i recommend decomposing the problem:
outer-loop for np.arange(0,10,0.01)-like fixing of variable
inner-loop for optimizing, where this variable is fixed
return the model with the best objective (with status=success)
This will effect in N inner-optimizations, where N=state-space of your to fix-var.
Depending on your task/data, it might be a good idea to traverse the fixing-space monotonically (like using np's arange) and use the solution of iteration i as initial-point for the problem i+1 (potentially less iterations needed if guess is good). But this is probably not relevant here, see next part.
If you really got 2 parameters, like indicated, this decomposition leads to an inner-problem with only 1 variable. Then, don't use minimize, use minimize_scalar (faster and more robust; does not need an initial-point).
Related
I am doing control engineering and I often face problems of the type below and I want to know if there is a way to deal with this in sympy.
question:
tl:dr: I want to make a MatrixSymbol dependent on a scalar Symbol representing time, to allow differentiation w.r.t. time.
Actual problem: v(t)=[v1(t),v2(t),v3(t)] is a vector function of the time t and I want to calculate the Projection into the direction of v and it's time derivative. In the end I would love to get an expression of v, v.diff(t) and v.T (the transpose).
attempts:
I've tried different things and show the closest one:
This does the algebra I need, but I cannot take derivatives w.r.t. time
v = MatrixSymbol('v',3,1)
# here i'm building the terms I want
projection_v = v*sqrt(v.T*v).inverse()*v.T
orthogonal_v = Identity(3)-projection_v
orthogonal_v.as_explicit()
orthogonal_v shows the abstract equation form that I need. In the end - to check and see the result again, I'd also like to make it explicit and see the expression as a function of v[0,0], v[1,0], and v[2,0] for MatrixSymbol the function .as_explicit() does exactly that beginning with sympy version 1.10. (Thanks Francesco Bonazzi for pointing this out.)
The problem however is, that I cannot make these a function of t and take the derivative of projection_v w.r.t. the time t.
I also tried
t = Symbol('t',real=True,positive=True)
v1 = Function('v1',real=True)(t)
v2 = Function('v2',real=True)(t)
v3 = Function('v3',real=True)(t)
v_mat = FunctionMatrix(3,1,[v1,v2,v3]);
but it seems FunctionMatrix is meant to evaluate the functions directly instead of being an analog to the scalar Function.
Effectively I want to be able to calculate orthogonal_v.diff(t) and then see the component wise operations with something like orthogonal_v.diff(t).as_explicit(). Is this possible?
I have a function of two variables, R(t,r), that has been constructed using a list of values for R, t, and r. This function cannot be written down, the values are found from solving a differential equation (d R(t,r)/dt). I require to take the derivatives of the function, in particular, I need
dR(t,r)/dr, d^2R(t,r)/drdt. I have tried using this answer to do this, but I cannot seem to get an answer that makes sense. (note that all derivatives should be partials). Any help would be appreciated.
Edit:
my current code. I understand getting anything to work without the `Rdata' file is impossible but the file itself is 160x1001. Really, any data could be made up to get the rest to work. Z_t does not return answers that seem like the derivative of my original function based on what I know, therefore, I know it is not differentiating my function as I'd expect.
If there are numerical routines for using the array of data I do not mind, I simply need some way of figuring out the derivatives.
import numpy as np
from scipy import interpolate
data = np.loadtxt('Rdata.txt')
rvals = np.linspace(1,160,160)
tvals = np.linspace(0,1000,1001)
f = interpolate.interp2d(tvals, rvals, data)
Z_t = interpolate.bisplev(tvals, rvals, f.tck, dx=0.8, dy=0)
I'm having trouble specifying constraints using basinhopping with method='COBYLA'. Here is a test case where things go wrong. Essentially, the constraints are ignored and there are function trials outside the specified range. I specify a simple quadratic with minimum at [0,0], searching for -3<x[0], but as you can see from the output, there are lots of searches outside that range (I increased the stepsize to make it obvious)
import numpy as np
from scipy.optimize import basinhopping
def f(x):
if x[0]<-3 :
print('outside range ',x[0])
return x[0]**2+x[1]**2
cons = [{'type':'ineq','fun': lambda x: x[0]+3}]
kwargs = {'method':'COBYLA','constraints':cons}
ret=basinhopping(f, [5,1],T=1,stepsize=1000,niter=1,minimizer_kwargs=kwargs)
print(ret)
runfile('py/cobyla_test', wdir='/py', post_mortem=True)
outside range -446.14581341127945
outside range -445.14581341127945
outside range -445.14581341127945
outside range -444.14581341127945
[etc... lots of output deleted]
[-4.81217825e-05 -5.23242054e-05] 5.0535284302996725e-09
As written at scipy.optimize.basinhopping — SciPy v1.1.0 Reference Guide, Basin-hopping is a two-step method:
first, a random jump is done (take_step callback)
then a local minimum is found from that point using the specificed minimization method
finally, it's decided if the step is accepted (accept_test callback)
The constraints you've specified are for the minimization method, they don't affect the jump step. For the jump step, either adjust stepsize (the maximum displacement for the random jump), or define your own take_step.
"I thought the point of the constraint is that it would never try an x outside the constraint" -- constraints in mathematical problems, including a constrained optimization problem, don't work that way. They only specify what conditions the solution itself must satisfy. They don't limit what points can be used while obtaining that solution, it's completely up to the algorithm to choose these.
The approach to limit the area in which a numerical method searches is to tweak method parameters in some way specific to the nature of the function and the method, to "guide" the method into the right direction.
I am running a non-linear optimization problem in OpenMDAO, which I know the optimal solution of (I just want to verify the solution). I am using SLSQP driver configuration of ScipyOptimizer from openmdao.api.
I have 3 design variables A, B and C, their respective design-spaces (Amin to Amax for A and so on) and a single objective function Z. As I said, I know the optimal values for all the three design variables (let's call them Asol, Bsol and Csol) which yield the minimum value of Z (call it Zsol).
When I run this problem, I get a value for Z which is larger than Zsol, signifying that it is not an optimal solution. When I assign Csol to C and run the problem with only A and B as the design variables, I get the value of Z which is much closer to Zsol and which is actually lesser than what I got earlier (in 3 design variable scenario).
Why am I observing this behavior? Shouldn't ScipyOptimizer give the same solution in both the cases?
EDIT: Adding some code..
from openmdao.api import IndepVarComp, Group, Problem
from openmdao.api import ScipyOptimizer
class RootGroup(Group):
def __init__(self):
super(RootGroup, self).__init__()
self.add('desvar_f', IndepVarComp('f', 0.08))
self.add('desvar_twc', IndepVarComp('tool_wear_compensation', 0.06))
self.add('desvar_V', IndepVarComp('V', 32.0))
# Some more config (adding components, connections etc.)
class TurningProblem_singlepart(Problem):
def __init__(self):
super(TurningProblem_singlepart, self).__init__()
self.root = RootGroup()
self.driver = ScipyOptimizer()
self.driver.options['optimizer'] = 'SLSQP'
self.driver.add_desvar('desvar_f.f', lower=0.08, upper=0.28)
self.driver.add_desvar('desvar_twc.tool_wear_compensation', lower=0.0, upper=0.5)
self.driver.add_desvar('desvar_V.V', lower=32.0, upper=70.0)
self.driver.add_objective('Inverse_inst.comp_output')
# Other config
This code gives me incorrect result. When I remove desvar_twc from both the classes, and assign it with its optimal value (from the solution I have), I get fairly correct result i.e. the answer for objective function which is lesser than the previous scenario.
Without seeing your actual model, we can't say anything for sure. However, it is NOT the case that a local optimizer's solution is independent of the starting condition in general. That is only case if the problem is convex. So I would guess that your problem is not convex, and you'r running into local optima.
You can try to get around this by using the COBYLA optimizer instead of SLSQP, which in my experience can manage to jump over some local optima better. But if your problem is really bumpy, then I would suggest you switch to NSGA-II or ALPSO from the pyopt-sparse library. These are heuristic based optimizers that do a good job of finding the "biggest hill", though they don't always climb all the way to the top of it (they don't converge as tightly). The heuristic algorithms are also generally more expensive than the gradient based methods.
I face a problem in scipy 'leastsq' optimisation routine, if i execute the following program it says
raise errors[info][1], errors[info][0]
TypeError: Improper input parameters.
and sometimes index out of range for an array...
from scipy import *
import numpy
from scipy import optimize
from numpy import asarray
from math import *
def func(apar):
apar = numpy.asarray(apar)
x = apar[0]
y = apar[1]
eqn = abs(x-y)
return eqn
Init = numpy.asarray([20.0, 10.0])
x = optimize.leastsq(func, Init, full_output=0, col_deriv=0, factor=100, diag=None, warning=True)
print 'optimized parameters: ',x
print '******* The End ******'
I don't know what is the problem with my func optimize.leastsq() call, please help me
leastsq works with vectors so the residual function, func, needs to return a vector of length at least two. So if you replace return eqn with return [eqn, 0.], your example will work. Running it gives:
optimized parameters: (array([10., 10.]), 2)
which is one of the many correct answers for the minimum of the absolute difference.
If you want to minimize a scalar function, fmin is the way to go, optimize.fmin(func, Init).
The issue here is that these two functions, although they look the same for a scalars are aimed at different goals. leastsq finds the least squared error, generally from a set of idealized curves, and is just one way of doing a "best fit". On the other hand fmin finds the minimum value of a scalar function.
Obviously yours is a toy example, for which neither of these really makes sense, so which way you go will depend on what your final goal is.
Since you want to minimize a simple scalar function (func() returns a single value, not a list of values), scipy.optimize.leastsq() should be replaced by a call to one of the fmin functions (with the appropriate arguments):
x = optimize.fmin(func, Init)
correctly works!
In fact, leastsq() minimizes the sum of squares of a list of values. It does not appear to work on a (list containing a) single value, as in your example (even though it could, in theory).
Just looking at the least squares docs, it might be that your function func is defined incorrectly. You're assuming that you always receive an array of at least length 2, but the optimize function is insanely vague about the length of the array you will receive. You might try writing to screen whatever apar is, to see what you're actually getting.
If you're using something like ipython or the python shell, you ought to be getting stack traces that show you exactly which line the error is occurring on, so start there. If you can't figure it out from there, posting the stack trace would probably help us.