I am looking into using Nlopt for solving optimisation problems in Python.
I have a series of simultaneous equations of the form
Ax = b
where A is an NxM matrix, with x the solution. Another way to think about this is that I have N simultaneous equations of the form x_1c_1m + x_2c_2m + .... + x_Nc_Nm = k_M, where x_i are variables to solve for, c_im is a constant associated with x_i when in equation M=m, and k_m is some constant in equation M=m. c_im and k_m are all known.
What confuses me is how to even approach this in Nlopt. Nlopt requires you to have actual callable functions, which I don't have? I suppose I could generalise each of the equations in that matrix equation above to something like:
def fn(x,c_m,k_m):
val = 0
for x_i, c_im in zip(x,c_m):
val += x_i * c_im
return val - k_m
where c_m and k_m would be already known, with the variables to solve for in x. All the examples I've seen have only been looking at a single variable problem, which has kind of thrown me a little. Would I then have to somehow define M copies of this function, and set each copy of fn as an equality constraint in the Nlopt optimisation object? It's all rather confusing. I'm looking to solve for x, which itself has multiple solutions, and I want to try to find the minimum values of x (or atleast an approximate solution if an exact solution cannot be found). Would I have to then set multiple objective functions, ie obj_fn_i = min(x_i) or something like that? It's all a little confusing to me in terms of what needs to be presented to the solver. I've already got an analytical solution to the above problem, so I can check my results reliably. Any help appreciated.
Cheers!
I have been using NLopt for a couple of problems, and what I have come to understand is the solver requires an objective function which returns a float value, so you must set the function as an MSE sum, or still as a single float value to be minimized. And it can solve for an array of variables x, in which both the objective function and constraint must depend. All equations that are involved in the system you can insert either in the objective function directly, or as constraints.
Hope this was helpful somehow!
Related
I have a very complicated function of two variables, let's call them x and y. I want to create a Python program where the user can input two values, a and b, where a is the value of that complicated function of x and y, and b = math.atan(y/x). This program should then output the values of x and y.
I am clueless as to where to start. I have tried to make the function into that of just one variable, then generate many random values for x and pick the closest one, but I have learnt that this is horribly inefficient and produces a result which is only accurate to about 2 significant figures, which is pretty horrible. Is there a better way to do this? Many thanks!
(P.S. I did not reveal the function here due to copyright issues. For the sake of example, you can consider the function
a = 4*math.atan(math.sqrt(math.tan(x)*math.tan(y)/math.tan(x+y)))
where y = x * math.tan(b).)
Edit: After using the approach of the sympy library, it appears as though the program ignores my second equation (the complicated one). I suspect it is too complicated for sympy to handle. Thus, I am asking for another approach which does not utilise sympy.
You could use sympy and import the trigonometric functions from sympy.
from sympy.core.symbol import symbols
from sympy.solvers.solveset import nonlinsolve
from sympy import sqrt, tan, atan
y = symbols('y', real=True)
a,b = 4,5 # user-given values
eq2 = a - 4*atan(sqrt(tan(y/tan(b))*tan(y)/tan((y/tan(b))+y)))
S = nonlinsolve( [eq2], [y] )
print(S)
It'll return you a series of conditions ( ConditionSet object ) for possible adequate results.
If that wasn't clear enough, you can read the docs for nonlinsolve.
I'm looking to set up a constraint-check in Python using PULP. Suppose I had variables A1,..,Xn and a constraint (AffineExpression) A1X1 + ... + AnXn <= B, where A1,..,An and B are all constants.
Given an assignment for X (e.g. X1=1, X2=4,...Xn=2), how can I check if the constraints are satisfied? I know how to do this with matrices using Numpy, but wondering if it's possible to do using PULP to let the library handle the work.
My hope here is that I can check specific variable assignments. I do not want to run an optimization algorithm on the problem (e.g. prob.solve()).
Can PULP do this? Is there a different Python library that would be better? I've thought about Google's OR-Tools but have found the documentation is a little bit harder to parse through than PULP's.
It looks like this is possible doing the following:
Define PULP variables and constraints and add them to an LpProblem
Make a dictionary of your assignments in the form {'variable name': value}
Use LpProblem.assignVarsVals(your_assignment_dict) to assign those values
Run LpProblem.valid() to check that your assignment meets all constraints and variable restrictions
Note that this will almost certainly be slower than using numpy and Ax <= b. Formulating the problem might be easier, but performance will suffer due to how PULP runs these checks.
You can stay in numpy and accomplish this. Looking at a single line from a matrix you can set your row of A equal to a vector and then create a row sum that allows you to check the index and find if it is true. For example:
a = A[0, :]
row_sum = a*x
sum(row_sum) <= B[0]
The last line will return just True or False. Then if you want to change a single index you could update your row_sum array by using
row_sum[3] = a[3]*new_val
and run your analysis again.
I am using a scipy.minimize function, where I'd like to have one parameter only searching for options with two decimals.
def cost(parameters,input,target):
from sklearn.metrics import mean_squared_error
output = self.model(parameters = parameters,input = input)
cost = mean_squared_error(target.flatten(), output.flatten())
return cost
parameters = [1, 1] # initial parameters
res = minimize(fun=cost, x0=parameters,args=(input,target)
model_parameters = res.x
Here self.model is a function that performs some matrix manipulation based on the parameters. Input and target are two matrices. The function works the way I want to, except I would like to have parameter[1] to have a constraint. Ideally I'd just like to give an numpy array, like np.arange(0,10,0.01). Is this possible?
In general this is very hard to do as smoothness is one of the core-assumptions of those optimizers.
Problems where some variables are discrete and some are not are hard and usually tackled either by mixed-integer optimization (working good for MI-linear-programming, quite okay for MI-convex-programming although there are less good solvers) or global-optimization (usually derivative-free).
Depending on your task-details, i recommend decomposing the problem:
outer-loop for np.arange(0,10,0.01)-like fixing of variable
inner-loop for optimizing, where this variable is fixed
return the model with the best objective (with status=success)
This will effect in N inner-optimizations, where N=state-space of your to fix-var.
Depending on your task/data, it might be a good idea to traverse the fixing-space monotonically (like using np's arange) and use the solution of iteration i as initial-point for the problem i+1 (potentially less iterations needed if guess is good). But this is probably not relevant here, see next part.
If you really got 2 parameters, like indicated, this decomposition leads to an inner-problem with only 1 variable. Then, don't use minimize, use minimize_scalar (faster and more robust; does not need an initial-point).
I'm currently trying to solve numerically a minimization problem and I tried to use the optimization library available in SciPy.
My function and derivative are a bit too complicated to be presented here, but they are based on the following functions, the minimization of which do not work either:
def func(x):
return np.log(1 + np.abs(x))
def grad(x):
return np.sign(x) / (1.0 + np.abs(x))
When calling the fmin_bfgs function (and initializing the descent method to x=10), I get the following message:
Warning: Desired error not necessarily achieved due to precision loss.
Current function value: 2.397895
Iterations: 0
Function evaluations: 24
Gradient evaluations: 22
and the output is equal to 10 (i.e. initial point). I suppose that this error may be caused by two problems:
The objective function is not convex: however I checked with other non-convex functions and the method gave me the right result.
The objective function is "very flat" when far from the minimum because of the log.
Are my suppositions true? Or does the problem come from anything else?
Whatever the error can be, what can I do to correct this? In particular, is there any other available minimization method that I could use?
Thanks in advance.
abs(x) is always somewhat dangerous as it is non-differentiable. Most solvers expect problems to be smooth. Note that we can drop the log from your objective function and then drop the 1, so we are left with minimizing abs(x). Often this can be done better by the following.
Instead of min abs(x) use
min t
-t <= x <= t
Of course this requires a solver that can solve (linearly) constrained NLPs.
I did this little test program in python to see how solve and nsolve work.
from sympy import *
theta = Symbol('theta')
phi = Symbol('phi')
def F(theta,phi):
return sin(theta)*cos(phi)+cos(phi)**2
def G(phi):
return ((1 + sqrt(3))*sin(phi) - 4*pi*sin(2*phi)*cos(2*phi))
solution1 = solve(F(pi/2,phi),phi)
solution2 = solve(G(phi),phi)
solution3 = nsolve(G(phi),0)
solution4 = nsolve(G(phi),1)
solution5 = nsolve(G(phi),2)
solution6 = nsolve(G(phi),3)
print solution1, solution2, solution3, solution4, solution5, solution6
And I get this output:
[pi/2, pi] [] 0.0 -0.713274788952698 2.27148961717279 3.14159265358979
The first call of solve gave me two solutions of the corresponding function. But not the second one. I wonder why? nsolve seems to work with an initial test value, but depending on that value, it gives different numerical solutions. Is there a way to get the list all numerical solutions with nsolve or with another function, in just one line?
The first call of solve gave me two solutions of the corresponding function. But not the second one. I wonder why?
In general, you cannot solve an equation symbolically and apparently solve does exactly that. In other words: Consider yourself lucky if solve can solve your equation, the typical technical applications don't have analytic solutions, that is, cannot be solved symbolically.
So the fall-back option is to solve the equation numerically, which starts from an initial point. In the general case, there is no guarantee that nsolve will find a solution even if exists one.
Is there a way to get the list all numerical solutions with nsolve or with another function, in just one line?
In general, no. Nevertheless, you can start nsolve from a number of initial guesses and keep track of the solutions found. You might want to distribute your initial guesses uniformly in the interval of interest. This is called multi-start method.