I'm looking for the fastest solution, x, to this polynomial equation:
Let m be an element in set M.
sum over all m {a_m * x^(b_m) - c_m * x^(b_m - 1)} = 0, where a_m, b_m, c_m are all different for each m. The set M has ~15-20 elements.
If the solution is > 4, it will return 4. If the solution is < 0, it will return 0.
What is the fastest way to do this? Doing it numerically?
I would prefer a solution in python, and other languages only if it's very beneficial to switch.
Note this is the derivative of an objective function. I am just trying to maximize the objective function, so if there's a better way to do it aside from solving this polynomial, that would work too! The solution should be fairly fast, as I am trying to solve many of these objective functions.
If you're only looking for one root and not all roots, you can use Newton's Method, which I expect is reasonably fast for the polynomials you've described.
let f(x) = sum over all m {a*x^(b) - c*x^(b-1)}
then f'(x), the derivative of f(x), is the sum over all m {(a*b)*x^(b-1) - (c*(b-1))*x^(b-2)}.
def newton(f, fprime, firstguess, epsilon):
x = firstguess
while abs(f(x)) > epsilon:
x = x - (f(x) / fprime(x))
return x
This will return an approximate root to your polynomial. If it's not accurate enough, pass in a smaller epsilon until it is accurate enough.
Note that this function may diverge, and run forever, or throw a ZeroDivisionError. Handle with caution.
Related
i'm triyng to solve this equation using "nsolve" function. unfortunately, this error appears:
ValueError: Could not find root within given tolerance. (435239733.760000060718 > 2.16840434497100886801e-19)
Try another starting point or tweak arguments.
The code is:
import sympy
d=[0.3, 32.6, 33.4, 241.7, 396.2, 444.4, 480.8, 588.9, 1043.9, 1136.1, 1288.1, 1408.1, 1439.4, 1604.8]
N=len(d)
x = sympy.Symbol('x', real=True)
expr2 = sympy.Eq(d[13] + N * sympy.Pow(x, -1) - N * d[13] * sympy.Pow(1 - sympy.exp(-d[13] * N), -1), 0)
expr_2 = sympy.simplify(expr=expr2)
solution = sympy.nsolve(expr_2, -0.01)
s = round(solution, 6)
print(s)
The system you are trying to solve leads to large derivatives and very abrupt changes, including a singularity at x == 0. This is the graph of the equation (Using Mathematica).
Numerical solvers struggle with these functions because most of them assumes some amount of smoothness around the solution and can be confused around singularities. Almost all of them (I'm talking about solvers in general, not just SymPy) benefits from regularization or a reformulation of the problem.
I would suggest to simplify the equation by multiplying both sides by x, which would remove the division by x and lead to a smoother function (linear in this case), for which numerical solvers behave correctly.
With this reformulation, you should find that the solution is 0.000671064.
Moreover, I would also suggest to rescale the coefficients so that they are all in [-1,1]. This also generally helps solvers. In your case, it will find the solution easily since it is linear, but more complex equations might cause problems.
Suppose I have an objective function f(x) that is non-convex (x can be a vector), but once I add constraints it becomes a convex problem. To show what i mean, consider this trivial example: let f(x) = cos(x). Clearly, cos(x) is not convex, but if i only consider x in [0, pi/2], then the function is convex when restricting x to these values.
CVXPY does not accept such a problem because it does not satisfy DCP rules. One option, in the previous example, is to minimize f(x) + g(x) where g is an indicator function such that: g(x) = 0 for x in [0, pi/2] and g(x) = +infinity otherwise, but I don't know how to implement that in CVXPY. What can i use in Python to take advantage of the "convexity" of the problem?
Thanks.
I have some data of the form
x1[i], x2[i], x3[i], z[i],
where z[i] is an unknown deterministic function of x1[i], x2[i], and x3[i]. I would like to find a quadratic function u(x1, x2, x3)= a11*x1^2 + a22*x2^2 + a33*x3^2 + a12*x1*x2 + ... + a0 that overbounds the data, i.e., u(x1[i], x2[i], x3[i]) >= z[i] for all i, and that minimizes the sum of the squared errors subject to the constraints.
Is there a computationally efficient solution approach in either Python or Matlab?
Your problem sounds like a quadratic programming problem with linear constraints. There are efficient algorithms to solve these and they are implemented in Matlab and Python as well; see quadprog and CVXOPT respectively.
There is very simple solution. Just use polynomial regression in Mathlab (http://www.matrixlab-examples.com/polynomial-regression.html).
You will get a certain function P(x1[i],x2[i],x3[i]).
1. Then for each i compute expression Diff[i] = P(x1[i],x2[i],x3[i]) - z[i].
You will get some array Diff.
2. Select all the negative values.
3. Find the minimum value in Diff: M = Min(Diff).
4. The desired function is F(x1[i],x2[i],x3[i]) = P(x1[i],x2[i],x3[i]) + Abs(M), where Abs(M) - it's value excluding the sign of M.
But if you're not just limited to quadratic functions, you can vary the degree of the polynomial and eventually get a more precise solution.
I have a function for with i need to do an infinite summation on (over all the integers) numerically. The summation doesn't always need to converge as I can change internal parameters. The function looks like,
m(g, x, q0) = sum(abs(g(x - n*q0))^2 for n in Integers)
m(g, q0) = minimize(m(g, x, q0) for x in [0, q0])
using a Pythonic pseudo-code
Using Scipy integration methods, I was just flooring the n and integrating like for a fixed x,
m(g, z, q0) = integrate.quad(lambda n:
abs(g(x - int(n)*q0))**2,
-inf, +inf)[0]
This works pretty well, but then I have to do optimization on the x as a function of x, and then do another summation on that which yields a integral of a optimization of an integral. Pretty much it takes a really long time.
Do you know of a better way to do the summation that is faster? Hand coding it seemed to go slower.
Currently, I am working with
g(x) = (2/sqrt(3))*pi**(-0.25)*(1 - x**2)*exp(-x**2/2)
but the solution should be general
The paper this comes from is "The Wavelet Transform, Time-Frequency Localization and Signal Analysis" by Daubechies (IEEE 1990)
Thank you
Thanks to all the useful comment, I wrote my own summator that seems to run pretty fast. It anyone has any recommendations to make it better, I will gladly take them.
I will test this on the problem I am working on and once it demonstrates success, I will claim it functional.
def integers(blk_size=100):
x = arange(0, blk_size)
while True:
yield x
yield -x -1
x += blk_size
#
# For convergent summation
# on not necessarily finite sequences
# processes in blocks which can be any size
# shape that the function can handle
#
def converge_sum(f, x_strm, eps=1e-5, axis=0):
total = sum(f(x_strm.next()), axis=axis)
for x_blk in x_strm:
diff = sum(f(x_blk), axis=axis)
if abs(linalg.norm(diff)) <= eps:
# Converged
return total + diff
else:
total += diff
g(x) is almost certainly your bottleneck. A very quick-and-dirty solution would be to vectorize it to operate on an array of integers, then use np.trapz to estimate the integral using the trapezoid rule:
import numpy as np
# appropriate range and step size depends on how accurate you need to be and how
# quickly the sum converges
xmin = -1000000
xmax = 1000000
dx = 1
x = np.arange(xmin, xmax + dx, dx)
gx = (2 / np.sqrt(3)) * np.pi**(-0.25)*(1 - x**2) * np.exp(-x**2 / 2)
sum_gx = np.trapz(gx, x, dx)
Aside from that, you could re-write g(x) using Cython or numba to speed it up.
There's a chance Numba improves speed significantly - http://numba.pydata.org
It's slightly painful to install but very easy to use. Have a look at:
https://jakevdp.github.io/blog/2015/02/24/optimizing-python-with-numpy-and-numba/
I'm trying to use the scipy Nelder-Mead simplex search function to find a minimum to a non-linear function. It appears my simplex gets stuck because it starts off with an initial simplex that is too small. Unfortunately, I don't see anywhere in scipy where you can change some of the simplex parameters (e.g. initial simplex size). Is there a way? Am I missing something? Or are there other implementations of the NM simplex?
Thanks
Two suggestions for Nelder-Mead:
1) snap all x to a grid, say .01, inside the function:
x = np.round( x / grid ) * grid
f = ...
This acts as a simple noise filter in high dimensions
(in 2d or 3d, don't bother).
2) start off with the best d+1 of 2d+1 nearby points,
instead of the usual d+1:
def neard1( func, x, h, verbose=1 ):
""" eval func at 2d+1 points x, x +- h
sort
-> f[ d+1 best values ], X[ d+1 ]
to start or restart Nelder-Mead
"""
dim = len(x)
I = np.eye(dim)
np.fill_diagonal( I, h ) # scalar or vec
X = x + np.vstack(( np.zeros(dim), I, - I ))
fnear = np.array([ func( x ) for x in X ]) # 2d+1
f0 = fnear[0]
up = np.argsort( fnear ) # vec func: |fnear|
if verbose:
print "neard1: f %g +- %s around x %s" % (
f0, fnear[up] - f0, x )
bestd1 = up[:dim+1]
return fnear[bestd1], X[bestd1]
It's also not a bad idea to look at the neard1() values after Nelder-Mead,
to get an idea of what func() looks like there.
If any neighbors are better then the N-M "best", restart N-M from that new simplex.
(One can alternate neard1, N-M, neard1, N-M: easy but very problem-dependent.)
How many variables do you have, and how noisy is your function ?
Hope this helps
From the reference at http://docs.scipy.org/doc/:
Method Nelder-Mead uses the Simplex algorithm [R123], [R124]. This algorithm has been successful in many applications but other algorithms using the first and/or second derivatives information might be preferred for their better performances and robustness in general.
It may be recommended to use a completely different algorithm, then. Note that:
Method BFGS uses the quasi-Newton method of Broyden, Fletcher, Goldfarb, and Shanno (BFGS) [R127] pp. 136. It uses the first derivatives only. BFGS has proven good performance even for non-smooth optimizations. This method also returns an approximation of the Hessian inverse, stored as hess_inv in the OptimizeResult object.
BFGS sounds more robust and faster overall.
ParagonRG