This is a general question about optimisation in Python. The problem I am attempting to solve has 91 parameters which need to be optimised in order to minimise the log likelihood function. Currently, I am using scipy.optimize.minimize, however, I find the process is very slow, compared to using Excel's solver, despite my expectation that Python would be much faster.
Is this a case of scipy.optimize.minimize isn't as fast as Excel's Solver in general or is the number of parameters too great for scipy? And if so, is there an alternative to Excel's Solver in Python that would be faster?
Related
Is there any way to force 'hybr' method of scipy.optimize 'root' to keep working even after it finds that convergence its too slow? In my problem, the solver nearly reaches desired precision, but right before it, the algorithm terminates because of slow convergence... Is it possible to make 'hybr' more 'self-confident'?
I use the root-finding algorithm root from scipy.optimize module to solve a system of two algebraic, non-linear equations. Since the equations have to be solved many times for various parameter values it is important to find a numerical method that would be most stable for this problem.
I have compared the performance of all the methods provided by scipy.optimize module. To visualize their performance I have used the following procedure:
The algebraic equations were rearranged so that they have zero on the R.H.S.
Then, at each step made by the algorithm, the sum of the L.H.S. squared of all the equations was computed and printed.
In my case, the most efficient method is the default "hybr". Other build-in methods either do not converge at all or are significantly slower. Unfortunately, in some cases the desired method gives up too fast. Lowering the precision and/or providing additional options to the functions did not help.
I have some intensive calculation to do involving hypergeomotric distribution and I'd like to know if there exists an implementation of this function that support 'real' vectorize calculation ? (Q1)
By 'real' I mean : not using the np.vectorize who seems to be implementing a for loop.
I know R implementation of the hypergeometric law allows for that kind of computation, which makes the code much more effcicient.
If it helps : I'm trying to duplicate an algorithm develop in R to Python, to compute exact confidence intervals for hypergeometric parameters. The concept was develop in the following paper :
http://www.wright.edu/~weizhen.wang/paper/37-2015jasa_wang.pdf
The solution involve iterating over all the possible combination of paremters to select the subset that satisfy to a specific condition.
(Q2) As an alternative, would it be possible to create my own udf using the definition of the law and the np.choose method but applied to vectors ?
(Q3) as a final option is it possible to call my script in R from Python ?
Thank you !
I am having trouble solving an optimisation problem in python, involving ~20,000 decision variables. The problem is non-linear and I wish to apply both bounds and constraints to the problem. In addition to this, the gradient with respect to each of the decision variables may be calculated.
The bounds are simply that each decision variable must lie in the interval [0, 1] and there is a monotonic constraint placed upon the variables, i.e each decision variable must be greater than the previous one.
I initially intended to use the L-BFGS-B method provided by the scipy.optimize package however I found out that, while it supports bounds, it does not support constraints.
I then tried using the SQLSP method which does support both constraints and bounds. However, because it requires more memory than L-BFGS-B and I have a large number of decision variables, I ran into memory errors fairly quickly.
The paper which this problem comes from used the fmincon solver in Matlab to optimise the function, which, to my knowledge, supports the application of both bounds and constraints in addition to being more memory efficient than the SQLSP method provided by scipy. I do not have access to Matlab however.
Does anyone know of an alternative I could use to solve this problem?
Any help would be much appreciated.
I have a constrained optimization problem for which I use the sp.optimize.minimize() function, with the SLSQP (Sequential Least Square Quadratic Programming) method.
A single access to the actual objective function is computationally quick. My problem is the minimize() routine does many fast access and then suddenly stops for a long time then does many fast iterations and waits and so on. So on the whole its slow, so is there anything I can do to alleviate this problem?
Any alternatives for constrained optimization other that SLSQP in scipy like PyOpt for example?
Usually I use Mathematica, but now trying to shift to python, so this question might be a trivial one, so I am sorry about that.
Anyways, is there any built-in function in python which is similar to the function named Interval[{min,max}] in Mathematica ? link is : http://reference.wolfram.com/language/ref/Interval.html
What I am trying to do is, I have a function and I am trying to minimize it, but it is a constrained minimization, by that I mean, the parameters of the function are only allowed within some particular interval.
For a very simple example, lets say f(x) is a function with parameter x and I am looking for the value of x which minimizes the function but x is constrained within an interval (min,max) . [ Obviously the actual problem is just not one-dimensional rather multi-dimensional optimization, so different paramters may have different intervals. ]
Since it is an optimization problem, so ofcourse I do not want to pick the paramter randomly from an interval.
Any help will be highly appreciated , thanks!
If it's a highly non-linear problem, you'll need to use an algorithm such as the Generalized Reduced Gradient (GRG) Method.
The idea of the generalized reduced gradient algorithm (GRG) is to solve a sequence of subproblems, each of which uses a linear approximation of the constraints. (Ref)
You'll need to ensure that certain conditions known as the KKT conditions are met, etc. but for most continuous problems with reasonable constraints, you'll be able to apply this algorithm.
This is a good reference for such problems with a few examples provided. Ref. pg. 104.
Regarding implementation:
While I am not familiar with Python, I have built solver libraries in C++ using templates as well as using function pointers so you can pass on functions (for the objective as well as constraints) as arguments to the solver and you'll get your result - hopefully in polynomial time for convex problems or in cases where the initial values are reasonable.
If an ability to do that exists in Python, it shouldn't be difficult to build a generalized GRG solver.
The Python Solution:
Edit: Here is the python solution to your problem: Python constrained non-linear optimization