Suppose you write a Python program to calculate the real roots of ax2 + bx + c = 0, where the coefficients a, b and c are given, real numbers. The traditional formulas for the two solutions are
x1 = (-b - sqrt(b*b-4*a*c)) / 2a,
x2 = (-b + sqrt(b*b-4*a*c)) / 2a.
I need to identify situations (values of the coefficients) where the formulas do not make sense or lead to large rounding errors, and propose alternative formulas that can be used in these cases in order to avoid a problem.
What do I need to do?
There are two situations where problems arise. The first is when the term inside the square root (the "Discriminant") goes negative, i.e.
if(b*b - 4*a*c < 0 ):
# do something. This doesn't have real roots
The second is more subtle. When you subtract two large numbers that are almost the same size, there is a possibility of rounding error. This will happen for the smaller root when 4*a*c << b*b. You could do a series expansion:
b - sqrt(b*b - 4*a*c)
= b * ( 1 - sqrt(1 - 4 * a * c / (b * b)))
~ b * ( 1 - 1 + 2 * a * c / (b * b)) # when 4*a*c << b*b
This term becomes
2 * a * c / b
So the final root would be
x1 = - c / b
Which is an interesting result. Of course the other root is still
x2 = (b + sqrt( b * b - 4 * a * c)) / (2 * a)
Not much error propagation there - although you could say that it will tend to
x2 = - b / a
when c becomes very small.
All this has nothing to do with Python - it is basic math. And I may have made errors - go ahead and see if you can find them.
For more help you might want to look at http://en.wikipedia.org/wiki/Quadratic_equation#Floating-point_implementation
Which provides a treatment of this issue in terms of numerical stability. You will find, among others, that the equations I (re)derived above are called "Vieta's formula".
Related
I have the following simple function to evaluate.
def f0(wt):
term1 = (1 + np.cos(wt)**2) * (1 / 3 - 2 / (wt)**2)
term2 = np.sin(wt)**2
term3 = 4 / (wt)**3 * np.cos(wt) * np.sin(wt)
return 0.5 * (term1 + term2 + term3)
For small values of wt (order of 1e-4 and below), I seem to have numerical problems in the evaluation of the function. Indeed, the term1 and term3 have very large and almost opposite values, but term2 is very small.
I think I improved things slightly by splitting the sum of the 3 terms into two parts, as showed here
def f1(wt):
# Split the calculation to have more stability hopefully
term1 = (1 + np.cos(wt)**2) * (1 / 3 - 2 / (wt)**2)
term2 = np.sin(wt)**2
term3 = 4 / (wt)**3 * np.cos(wt) * np.sin(wt)
partial = term1 + term3
return 0.5 * (partial + term2)
However, for very small but positive values of wt, I think there are still numerical problems. I expect this function to be smooth for any positive value of wt, but, as you can see from the plot attached, at values below 1e-3, there are wild artifacts.
My question is: how can I improve the numerical precision of Numpy, if I am already using the data type float64?
Note: I am on a Windows 10 machine with 64 bits. I have read on other Stack Overflow threads that the class np.float128 is not available.
Full code snippet
import numpy as np
import matplotlib.pyplot as plt
wt = np.logspace(-6, 1, 1000)
def f0(wt):
term1 = (1 + np.cos(wt)**2) * (1 / 3 - 2 / (wt)**2)
term2 = np.sin(wt)**2
term3 = 4 / (wt)**3 * np.cos(wt) * np.sin(wt)
return 0.5 * (term1 + term2 + term3)
def f1(wt):
# Split the calculation to have more stability hopefully
term1 = (1 + np.cos(wt)**2) * (1 / 3 - 2 / (wt)**2)
term2 = np.sin(wt)**2
term3 = 4 / (wt)**3 * np.cos(wt) * np.sin(wt)
partial = term1 + term3
return 0.5 * (partial + term2)
plt.figure()
plt.loglog(wt, f0(wt), label='f0')
plt.loglog(wt, f1(wt), label='f1')
plt.grid()
plt.legend()
plt.xlabel('wt')
plt.show()
How about you replace the sin and cosin with the first few terms of their Taylor series. Then sympy is able to give you a simple result that is hopefully better suited numerically.
First I slightly change your function so it gives me a sympy expression.
from sympy import *
t = symbols('t')
def f0(wt):
term1 = (1 + sympy.cos(wt)**2) * (sympy.Rational(1,3) - 2 / (wt)**2)
term2 = sympy.sin(wt)**2
term3 = 4 / (wt)**3 * sympy.cos(wt) * sympy.sin(wt)
return sympy.Rational(1,2)*(term1 + term2 + term3)
expr = f0(t)
expr
Now I replace sin and cos with their taylor polynomials.
def taylor(f, n):
return sum(t**i/factorial(i) * f(t).diff(t, i).subs(t,0) for i in range(n))
tsin = taylor(sin, 7)
tcos = taylor(cos, 7)
expr2 = simplify(expr.subs(sin(t),tsin).subs(cos(t),tcos))
f1 = lambdify(t, expr2, 'numpy')
expr2
And finally I plot it using exactly your code. Notice that I am using sympys option to make a numpy ufunc.
wt = np.logspace(-6, 1, 1000)
plt.figure()
plt.loglog(wt, f0(wt), label='f0')
plt.loglog(wt, f1(wt), label='f1')
plt.grid()
plt.legend()
plt.xlabel('wt')
plt.show()
Obviously this function is only good around zero and for values between 1 and 10 you should take the original function. But in case you need convincing and don't care that the function with replaced taylor polynomial looks nice you can crank the degree up to 25 making it visually agree with your function at least up until 10.
And you can combine the functions so it calculates the values around zero with my function and the other with yours like this.
def f2(wt):
cond = np.abs(wt) > 1/10
return np.piecewise(wt, [cond, ~cond], [f0,f1])
The problem you are facing is catastrophic cancellation and it must not be solved using higher precision as doing so will generally postpone the actual problem. The root of the problem which is a numerical instability must be solved by reformulating the mathematical expression.
Note that f1 is a bit better than f0 but the cancellation issue lies in term1 + term3.
By transforming the expression simple development/factorization operations and using trigonometric identities one can get the following function:
def f2(wt):
sw = np.sin(wt)
sw2 = np.sin(2*wt)
return (sw/wt)**2 + 1/3 + (sw2 / wt - 2) / wt**2 + sw**2 / 3
This function is a bit more accurate but still contains a cancellation causing the same issue. This happens because of the expression E = (sw2 / wt - 2) / wt**2 which is the root of the problem. Indeed, np.sin(2*wt) tends towards 2 when wt is near 0. Thus sw2 / wt - 2 is close to 0 and the expression E is numerically unstable because of a close-to-zero value divided by another close-to-zero value. If one can reformulate analytically E to remove the singularity, then the resulting expression will likely be numerically stable. For more information you can look at the sinc function and how to compute an approximation of this function (also available in Numpy).
One simple way to solve this is to use numerical tools like Taylor series. Taylor series can approximate the expression of E close to zero accurately (because of its derivatives). Actually, one can use Taylor series to compute the whole expression and not only E. However, using Taylor series for values close to 1 give inaccurate results. In fact, the accuracy of the method drops very quickly above 1. One solution is to only use the Taylor series for small values.
Here is the resulting implementation:
def f3(wt):
sw = np.sin(wt)
sw2 = np.sin(2*wt)
reference = (sw/wt)**2 + 1/3 + (sw2 / wt - 2) / wt**2 + sw**2 / 3
# O(13) Taylor series computation used only for near-zero values
taylor = ( ( 4. / 15.) * wt**2
- ( 29. / 315.) * wt**4
+ ( 37. / 2835.) * wt**6
- (151. / 155925.) * wt**8
+ (268. / 6081075.) * wt**10
- (866. / 638512875.) * wt**12)
# Select the best implementation
return np.where(np.logical_and(wt >= -0.2, wt <= 0.2), taylor, reference)
This implementation appear to be very accurate in practice (>=12 digits of precision) while being still relatively fast. Here is the result:
I have small problem with too big float numbers in Python. Overflow errors occur.
The function look like this:
N(t + 1) = (1 + c) N(t) - (c / d) * N(t)**2 where t is time, and c and d are constants.
I need it to calculate up to one minute (t = 60). But after t= 8, 9 or 10 I got overflow error.
def nValue(t):
n_calc = []
c = 2.3 # 0..3
d = 95 # 1..250
n_0 = 450 # 0..600
n_1 = (1 + c) * n_0 - c / d * n_0**2
n_calc.append(n_0)
n_calc.append(n_1)
for i in range(2, t+1):
n_curr = (1 + c) * n_calc[i- 1] - (c / d) * (n_calc[i- 1]**2)
n_calc.append(n_curr)
return n_calc
Should I use Decimal type, BigFloat or something different?
Finally I have to draw plot... (2D Matplotlib plot). Maybe I should "scale" somehow this equation?
You could just use this previous answer as reference.
OverflowError: (34, 'Result too large')
Decimal class has no limit so you should use use. It serves the same purpose as GMP and other arbitrary precision libraries.
I hope this helps you.
https://en.wikipedia.org/wiki/List_of_arbitrary-precision_arithmetic_software
How do I simplify a*sin(wt) + b*cos(wt) into c*sin(wt+theta) using SymPy? For example:
f = sin(t) + 2*cos(t) = 2.236*sin(t + 1.107)
I tried the following:
from sympy import *
t = symbols('t')
f=sin(t)+2*cos(t)
trigsimp(f) #Returns sin(t)+2*cos(t)
simplify(f) #Returns sin(t)+2*cos(t)
f.rewrite(sin) #Returns sin(t)+2*sin(t+Pi/2)
PS.: I dont have direct access to a,b and w. Only to f
Any suggestion?
The general answer can be achieved by noting that you want to have
a * sin(t) + b * cos(t) = A * (cos(c)*sin(t) + sin(c)*cos(t))
This leads to a simultaneous equation a = A * cos(c) and b = A * sin(c).
Dividing the second equation by the second, we can solve for c. Substituting its solution into the first equation, you can solve for A.
I followed the same pattern but just to get it in terms of cos. If you want to get it in terms of sin, you can use Rodrigo's formula.
The following code should be able to take any linear combination of the form x * sin(t - w) or y * cos(t - z). There can be multiple sins and cos'.
from sympy import *
t = symbols('t', real=True)
expr = sin(t)+2*cos(t) # unknown
d = collect(expr.expand(trig=True), [sin(t), cos(t)], evaluate=False)
a = d[sin(t)]
b = d[cos(t)]
cos_phase = atan(a/b)
amplitude = a / sin(cos_phase)
print(amplitude.evalf() * cos(t - cos_phase.evalf()))
Which gives
2.23606797749979*cos(t - 0.463647609000806)
This seems to be a satisfactory match after plotting both graphs.
You could even have something like
expr = 2*sin(t - 3) + cos(t) - 3*cos(t - 2)
and it should work fine.
a * sin(wt) + b * cos(wt) = sqrt(a**2 + b**2) * sin(wt + acos(a / sqrt(a**2 + b**2)))
While the amplitude is the radical sqrt(a**2 + b**2), the phase is given by the arccosine of the ratio a / sqrt(a**2 + b**2), which may not be expressible in terms of arithmetic operations and radicals. Hence, you may be asking SymPy to do the impossible. Better use floating-point values, but you do not need SymPy for that.
I'm attempting to implement Vincenty's inverse problem as described on wiki HERE
The problem is that lambda is simply not converging. The value stays the same if I try to iterate over the sequence of formulas, and I'm really not sure why. Perhaps I've just stared myself blind on an obvious problem.
It should be noted that I'm new to Python and still learning the language, so I'm not sure if it's misuse of the language that might cause the problem, or if I do have some mistakes in some of the calculations that I perform. I just can't seem to find any mistakes in the formulas.
Basically, I've written in the code in as close of a format as I could to the wiki article, and the result is this:
import math
# Length of radius at equator of the ellipsoid
a = 6378137.0
# Flattening of the ellipsoid
f = 1/298.257223563
# Length of radius at the poles of the ellipsoid
b = (1 - f) * a
# Latitude points
la1, la2 = 10, 60
# Longitude points
lo1, lo2 = 5, 150
# For the inverse problem, we calculate U1, U2 and L.
# We set the initial value of lamb = L
u1 = math.atan( (1 - f) * math.tan(la1) )
u2 = math.atan( (1 - f) * math.tan(la2) )
L = (lo2 - lo1) * 0.0174532925
lamb = L
while True:
sinArc = math.sqrt( math.pow(math.cos(u2) * math.sin(lamb),2) + math.pow(math.cos(u1) * math.sin(u2) - math.sin(u1) * math.cos(u2) * math.cos(lamb),2) )
cosArc = math.sin(u1) * math.sin(u2) + math.cos(u1) * math.cos(u2) * math.cos(lamb)
arc = math.atan2(sinArc, cosArc)
sinAzimuth = ( math.cos(u1) * math.cos(u2) * math.sin(lamb) ) // ( sinArc )
cosAzimuthSqr = 1 - math.pow(sinAzimuth, 2)
cosProduct = cosArc - ((2 * math.sin(u1) * math.sin(u2) ) // (cosAzimuthSqr))
C = (f//16) * cosAzimuthSqr * (4 + f * (4 - 3 * cosAzimuthSqr))
lamb = L + (1 - C) * f * sinAzimuth * ( arc + C * sinArc * ( cosProduct + C * cosArc * (-1 + 2 * math.pow(cosProduct, 2))))
print(lamb)
As mentioned the problem is that the value "lamb" (lambda) will not become smaller. I've even tried to compare my code to other implementations, but they looked just about the same.
What am I doing wrong here? :-)
Thank you all!
First, you should convert you latitudes in radians too (you already do this for your longitudes):
u1 = math.atan( (1 - f) * math.tan(math.radians(la1)) )
u2 = math.atan( (1 - f) * math.tan(math.radians(la2)) )
L = math.radians((lo2 - lo1)) # better than * 0.0174532925
Once you do this and get rid of // (int divisions) and replace them by / (float divisions), lambda stops repeating the same value through your iterations and starts following this path (based on your example coordinates):
2.5325205864224847
2.5325167509030906
2.532516759118641
2.532516759101044
2.5325167591010813
2.5325167591010813
2.5325167591010813
As you seem to expect a convergence precision of 10^(−12), it seems to make the point.
You can now exit the loop (lambda having converged) and keep going until you compute the desired geodesic distance s.
Note: you can test your final value s here.
Even if it is correctly implemented, Vincenty's algorithm will fail to
converge for some points. (This problem was noted by Vincenty.)
I give an algorithm which is guaranteed to
converge in Algorithms for geodesics; there's a python
implementation available here. Finally, you can find more
information on the problem at the Wikipedia page,
Geodesics on an ellipsoid. (The talk page has examples
of pairs of points for which Vincenty, as implemented by the NGS,
fails to converge.)
I'd like to implement Euler's method (the explicit and the implicit one)
(https://en.wikipedia.org/wiki/Euler_method) for the following model:
x(t)' = q(x_M -x(t))x(t)
x(0) = x_0
where q, x_M and x_0 are real numbers.
I know already the (theoretical) implementation of the method. But I couldn't figure out where I can insert / change the model.
Could anybody help?
EDIT: You were right. I didn't understand correctly the method. Now, after a few hours, I think that I really got it! With the explicit method, I'm pretty sure (nevertheless: could anybody please have a look at my code? )
With the implicit implementation, I'm not very sure if it's correct. Could please anyone have a look at the implementation of the implicit method and give me a feedback what's correct / not good?
def explizit_euler():
''' x(t)' = q(xM -x(t))x(t)
x(0) = x0'''
q = 2.
xM = 2
x0 = 0.5
T = 5
dt = 0.01
N = T / dt
x = x0
t = 0.
for i in range (0 , int(N)):
t = t + dt
x = x + dt * (q * (xM - x) * x)
print '%6.3f %6.3f' % (t, x)
def implizit_euler():
''' x(t)' = q(xM -x(t))x(t)
x(0) = x0'''
q = 2.
xM = 2
x0 = 0.5
T = 5
dt = 0.01
N = T / dt
x = x0
t = 0.
for i in range (0 , int(N)):
t = t + dt
x = (1.0 / (1.0 - q *(xM + x) * x))
print '%6.3f %6.3f' % (t, x)
Pre-emptive note: Although the general idea should be correct, I did all the algebra in place in the editor box so there might be mistakes there. Please, check it yourself before using for anything really important.
I'm not sure how you come to the "implicit" formula
x = (1.0 / (1.0 - q *(xM + x) * x))
but this is wrong and you can check it by comparing your "explicit" and "implicit" results: they should slightly diverge but with this formula they will diverge drastically.
To understand the implicit Euler method, you should first get the idea behind the explicit one. And the idea is really simple and is explained at the Derivation section in the wiki: since derivative y'(x) is a limit of (y(x+h) - y(x))/h, you can approximate y(x+h) as y(x) + h*y'(x) for small h, assuming our original differential equation is
y'(x) = F(x, y(x))
Note that the reason this is only an approximation rather than exact value is that even over small range [x, x+h] the derivative y'(x) changes slightly. It means that if you want to get a better approximation of y(x+h), you need a better approximation of "average" derivative y'(x) over the range [x, x+h]. Let's call that approximation just y'. One idea of such improvement is to find both y' and y(x+h) at the same time by saying that we want to find such y' and y(x+h) that y' would be actually y'(x+h) (i.e. the derivative at the end). This results in the following system of equations:
y'(x+h) = F(x+h, y(x+h))
y(x+h) = y(x) + h*y'(x+h)
which is equivalent to a single "implicit" equation:
y(x+h) - y(x) = h * F(x+h, y(x+h))
It is called "implicit" because here the target y(x+h) is also a part of F. And note that quite similar equation is mentioned in the Modifications and extensions section of the wiki article.
So now going to your case that equation becomes
x(t+dt) - x(t) = dt*q*(xM -x(t+dt))*x(t+dt)
or equivalently
dt*q*x(t+dt)^2 + (1 - dt*q*xM)*x(t+dt) - x(t) = 0
This is a quadratic equation with two solutions:
x(t+dt) = [(dt*q*xM - 1) ± sqrt((dt*q*xM - 1)^2 + 4*dt*q*x(t))]/(2*dt*q)
Obviously we want the solution that is "close" to the x(t) which is the + solution. So the code should be something like:
b = (q * xM * dt - 1)
x(t+h) = (b + (b ** 2 + 4 * q * x(t) * dt) ** 0.5) / 2 / q / dt
(editor note:) Applying the binomial complement, this formula has the numerically more stable form for small dt, where then b < 0,
x(t+h) = (2 * x(t)) / ((b ** 2 + 4 * q * x(t) * dt) ** 0.5 - b)