Does anyone know which optimization algorithm specifically is implemented in scipy.optimize.leastsq?
According to the documentation:
“leastsq” is a wrapper around MINPACK’s lmdif and lmder algorithms.
So (as #tillsten points out) that’s a Levenberg–Marquardt implementation.
Related
I'm trying to fit my data and have so far used sp.optimize.leastsq. I changed to sp.optimize.least_squares to add bounds to the parameters, but both when I use bounds and when I don't the search doesn't converge, even in data sets sp.optimize.leastsq fitted just fine.
Shouldn't these functions work the same?
What could be the difference between them that makes the newer one not to find solutions the older one did?
leastsq
is a wrapper around MINPACK’s lmdif and lmder algorithms.
least_squares implements other methods in addition to the MINPACK algorithm.
method{‘trf’, ‘dogbox’, ‘lm’}, optional
Algorithm to perform minimization.
‘trf’ : Trust Region Reflective algorithm, particularly suitable for large sparse problems with bounds. Generally robust method.
‘dogbox’ : dogleg algorithm with rectangular trust regions, typical use case is small problems with bounds. Not recommended for problems with rank-deficient Jacobian.
‘lm’ : Levenberg-Marquardt algorithm as implemented in MINPACK. Doesn’t handle bounds and sparse Jacobians. Usually the most efficient method for small unconstrained problems.
Default is ‘trf’. See Notes for more information.
It is possible for some problems that lm method does not converge while trf converges.
So I want to try to solve my optimization problem using particle swarm optimiztion algorithm. As I comoratable with python I was looking into PySwarms toolkit. The issue is I am not really experienced in this field and don't really know how to account for integrality constraints of my problem. I was looking for advice on what are some approches to dealing with integral variables in PSO. And maybe some examples with PySwarms or any good alternative packages?
You can try pymoo module, which is an excellent multi-objective optimization tool. It can also solve mixed variable problems. Despite pymoo is first of all designed to solve such problems using genetic algorithms, there is an implementation of PSO (single-objective with continuous variables). Maybe you'll find it useful to try to solve your mixed variable problem using genetic algorithm or one of its modifications (e.g. NSGAII).
I'm learning python to make models these days. I read the documentation of scipy.optimize.fmin. It also recommends scipy.optimize.minimize. It seems that scipy.optimize.minimize is a more advanced method. Real wonder what's the difference between these two.
scipy.optimize.minimize is a high-level interface that lets you choose from a broad range of solvers, one of which is Nelder–Mead. scipy.optimize.fmin is a special solver using Nelder–Mead. For a specific subdocumentation of minimize with Nelder–Mead, see here.
I am using Pyomo to model my optimization problem (MILP) and solve it using Gurobi.
What would be the best, fastest or easiest way to find a heuristic solution using the Pyomo model, knowing that I do not care about the Gap bounds.
Note: I know that Gurobi has a heuristic solver but it doesn't tell what heuristic algorithm they are using!
Finding a heuristic solution to some MILP problem is complexity-wise as hard as optimizing it!
There is no best, fastest, easiest way in general. You always want to exploit some problem-characteristics.
As start, just use any MIP-solver and tune the params to reflect your needs. If you want just any heuristic solution, tune the solver for feasibility, probably meaning a higher frequency of heuristic-steps and early-stop with the first feasible solution.
Yes, you won't know what's Gurobi using internally. But knowing all of the code would not help much either. It's surely not something which you can find on wikipedia then (except for classic stuff like the feasibility pump or Relaxation induced neighborhood search).
If you want to know more about these methods, check out papers on MIP-heuristics in general! You will see, that most Heuristics are tightly coupled with the MIP-nature of the problem (although i expect some SAT-solver-usage internally too in commercial ones).
Does somebody know about a entropy minimization implementation in python?
I want to use this to correct inhomogeneity in histology images.
Reference about the algorithm I am talking about.
I suspect that you'll have to hack something together yourself, maybe using PIL and pyentropy.