I am trying to minimize a complex objective function that has 2 decision variables
The variables have bounds as mentioned below:
0<= var1/var2 < Some_upper_bound
As per my understanding of bounds variable in the optimize.minimize() function, both upper and lower values are inclusive.
How do I create a bound such that one value is inclusive (0) and the other is exclusive (Some_upper_bound)?
Any help with this is much appreciated. Thanks in advance!
No tool will do what you want. For a reason: the feasible region is no longer compact and the optimization problem is no longer well defined. Also, most solvers actually use a feasibility tolerance, so their constraint effectively becomes: x <= a + feastol. This is done because computations suffer from limited floating point precision. See: https://yetanothermathprogrammingconsultant.blogspot.com/2017/03/strict-inequalities-in-optimization.html.
Related
Pyomo provides some features to add constraints into variables like bellow code in the document.
model.LumberJack = Var(within=NonNegativeReals, bounds=(0,6), initialize=1.5)
But, I want to define a variable with open interval constraints such as (0, 1]. In my understanding, the bounds argument means closed interval, so, if I set the param as bounds=(0,1), it means [0, 1].
I think closed interval constraints are common things and Pyomo provide this kind of features, but I couldn't find it. Is it a implementation issue? or theoretical issues in optimization?
An open interval would mean a "strictly less" constraint in the model, i.e.
variable < upper bound
instead of
variable <= upper bound
Depending on your solution algorithm this may not be supported by the underlying theory. For example, in linear and mixed integer programming theory there is no support for strict inequalities. The only thing you can have is <= and >=.
So even if Pyomo would support (half-)open intervals, the algorithms to solve the problem may not.
The usual approach to work around this is to use a small epsilon and write
variable <= upper bound - epsilon
to "emulate" a strict inequality. This may of course introduce numerical difficulties.
Finally, given that most algorithms operate with finite precision and numerical tolerances, there is the question what a strict inequality on a variable bound should mean. As soon as the tolerance is bigger than 0 the variable would be allowed to attain the value at the upper bound and that would be considered feasible within tolerances.
I have a piece of code that I am using scipy.integrate.quad. The limits of integration are minus infinity to infinity. It runs OK, but I would like it faster.
The nature of the problem is that the function being integrated is the product of three functions: (1) one that is narrow (between zero and (2) one that is wide (between, say, 200,000 and 500,000), and (3) one that falls off as 1/abs(x).
I only need accuracy to .1%, if that.
I could do a lot of work and actually determine integration limits that are real numbers so no excess computation gets done; outside the regions of functions 1 and 2 they are both zero, so the 1/x doesn't even come into play there. But it would be a fair amount of error-prone code calculations.
How does this function know how to optimize, and is it pretty good at it, with infinite bounds?
Can I tune it through passing in guidance (like error tolerance)?
Or, would it be worthwhile to try to give it limited integration bounds?
quad uses different algorithms for finite and infinite intervals, but the general idea is the same: the integral is computed using two related methods (for example, 7-point Gauss rule and 15-point Kronrod rule), and the difference between those results provides an estimate for how accurate they are. If the accuracy is low, the interval is bisected and the process repeats for subintervals. A detailed explanation is beyond the scope of a Stack Overflow answer; numerical integration is complicated.
For large or infinite integration bounds, the accuracy and efficiency depend on the algorithm being able to locate the main features of the function. Passing the bounds as -np.inf, np.inf is risky. For example,
quad(lambda x: np.exp(-(x-20)**2), -np.inf, np.inf)
returns a wrong result (essentially zero instead of 1.77) because it does not notice the bump of the Gaussian function near 20.
On the other hand, arbitrarily imposing a finite interval is questionable in that you give up any control over error (no estimate on what was contained in the infinite tails that you cut off). I suggest the following:
Split the integral into three: (-np.inf, A), (A, B), and (B, np.inf) where, say, A is -1e6 and B is 1e6.
For the integral over (A, B), provide points parameter, which locates the features ("narrow parts") of the function. For example,
quad(lambda x: np.exp(-(x-20)**2), -1e6, 1e6, points=[10, 30])
returns 1.77 as it should.
Adjust epsabs (absolute error) and epsrel (relative error) to within desired accuracy, if you find that the default accuracy is too demanding.
I want to run an optimisation problem repeatedly to further refine the end result.
Essentially, the objective is to minimise the maximum of a set of variables (subject to inequality and equality constraints), and then minimise the maximum of the set excluding the maximum, and then minimise the maximum of the set excluding the two largest numbers and so on...
The algorithm I have in mind is:
Run scipy.linprog(..., bounds=[(-numpy.inf, numpy.inf), (-numpy.inf, numpy.inf), (-numpy.inf, numpy.inf), ...]) with all variables unbounded, to minimise the maximum of the set of numbers.
Assuming optimisation problem is feasible and successfully solved, fix the maximum to opt_val by setting bounds=[..., (opt_val, opt_val), ...], where all other variables have the bounds (-numpy.inf, numpy.inf).
Make inequality constraints corresponding to that variable ineffective, by changing the coefficient of b_ub to numpy.inf.
Rerun simulation with modified bounds and inequality vector.
This can run without error, but it seems like scipy/numpy explicitly ignore the bounds I place on the variables - I get results for the variables that I have 'fixed' that are not the corresponding opt_val.
Can scipy handle bounds that restrict a variable to a single floating point number?
Is this the best way to be solving my problem?
The code I have developed is really quite extensive, which is why I have not posted it here, so of course I don't expect a code-based solution. What I am looking for here is a yes/no answer as to whether scipy can handle bounding intervals restricted to a single float, and, on a higher level, whether I have the correct approach.
The documentation at https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.optimize.linprog.html does not explicitly say whether or not it is possible to specify fixed-point bounds.
It turns out it was a problem with relaxing the inequality constraints. I had mistakenly relaxed all constraints regarding the fixed variables, when instead I needed to have relaxed some of the constraints.
#ErwinKalvelagen's comment is still worth noting however.
I am attempting to calculate integrals between two limits using python/scipy.
I am using online calculators to double check my results (http://www.wolframalpha.com/widgets/view.jsp?id=8c7e046ce6f4d030f0b386ea5c17b16a, http://www.integral-calculator.com/), and my results disagree when I have certain limits set.
The code used is:
import scipy as sp
import numpy as np
def integrand(x):
return np.exp(-0.5*x**2)
def int_test(a,b):
# a and b are the lower and upper bounds of the integration
return sp.integrate.quad(integrand,a,b)
When setting the limits (a,b) to (-np.inf,1) I get answers that agree (2.10894...)
however if I set (-np.inf,300) I get an answer of zero.
On further investigation using:
for i in range(50):
print(i,int_test(-np.inf,i))
I can see that the result goes wrong at i=36.
I was wondering if there was a way to avoid this?
Thanks,
Matt
I am guessing this has to do with the infinite bounds. scipy.integrate.quad is a wrapper around quadpack routines.
https://people.sc.fsu.edu/~jburkardt/f_src/quadpack/quadpack.html
In the end, these routines chose suitable intervals and try to get the value of the integral through function evaluations and then numerical integrations. This works fine for finite integrals, assuming you know roughly how fine you can make the steps of the function evaluation.
For infinite integrals it depends how well the algorithms choose respective subintervals and how accurately they are computed.
My advice: do NOT use numerical integration software AT ALL if you are interested in accurate values for infinite integrals.
If your problem can be solved analytically, try that or confine yourself to certain bounds.
I'm trying to perform a constrained least-squares estimation using Scipy such that all of the coefficients are in the range (0,1) and sum to 1 (this functionality is implemented in Matlab's LSQLIN function).
Does anybody have tips for setting up this calculation using Python/Scipy. I believe I should be using scipy.optimize.fmin_slsqp(), but am not entirely sure what parameters I should be passing to it.[1]
Many thanks for the help,
Nick
[1] The one example in the documentation for fmin_slsqp is a bit difficult for me to parse without the referenced text -- and I'm new to using Scipy.
scipy-optimize-leastsq-with-bound-constraints on SO givesleastsq_bounds, which is
leastsq
with bound constraints such as 0 <= x_i <= 1.
The constraint that they sum to 1 can be added in the same way.
(I've found leastsq_bounds / MINPACK to be good on synthetic test functions in 5d, 10d, 20d;
how many variables do you have ?)
Have a look at this tutorial, it seems pretty clear.
Since MATLAB's lsqlin is a bounded linear least squares solver, you would want to check out scipy.optimize.lsq_linear.
Non-negative least squares optimization using scipy.optimize.nnls is a robust way of doing it. Note that, if the coefficients are constrained to be positive and sum to unity, they are automatically limited to interval [0,1], that is one need not additionally constrain them from above.
scipy.optimize.nnls automatically makes variables positive using Lawson and Hanson algorithm, whereas the sum constraint can be taken care of as discussed in this thread and this one.
Scipy nnls uses an old fortran backend, which is apparently widely used in equivalent implementations of nnls by other software.