I have been trying to create custom calculator for calculating trigonometric functions. Aside from Chebyshev pylonomials and/or Cordic algorithm I have used Taylor series which have been accurate by few places of decimal.
This is what i have created to calculate simple trigonometric functions without any modules:
from __future__ import division
def sqrt(n):
ans = n ** 0.5
return ans
def factorial(n):
k = 1
for i in range(1, n+1):
k = i * k
return k
def sin(d):
pi = 3.14159265359
n = 180 / int(d) # 180 degrees = pi radians
x = pi / n # Converting degrees to radians
ans = x - ( x ** 3 / factorial(3) ) + ( x ** 5 / factorial(5) ) - ( x ** 7 / factorial(7) ) + ( x ** 9 / factorial(9) )
return ans
def cos(d):
pi = 3.14159265359
n = 180 / int(d)
x = pi / n
ans = 1 - ( x ** 2 / factorial(2) ) + ( x ** 4 / factorial(4) ) - ( x ** 6 / factorial(6) ) + ( x ** 8 / factorial(8) )
return ans
def tan(d):
ans = sin(d) / sqrt(1 - sin(d) ** 2)
return ans
Unfortunately i could not find any sources that would help me interpret inverse trigonometric function formulas for Python. I have also tried putting sin(x) to the power of -1 (sin(x) ** -1) which didn't work as expected.
What could be the best solution to do this in Python (In the best, I mean simplest with similar accuracy as Taylor series)? Is this possible with power series or do i need to use cordic algorithm?
The question is broad in scope, but here are some simple ideas (and code!) that might serve as a starting point for computing arctan. First, the good old Taylor series. For simplicity, we use a fixed number of terms; in practice, you might want to decide the number of terms to use dynamically based on the size of x, or introduce some kind of convergence criterion. With a fixed number of terms, we can evaluate efficiently using something akin to Horner's scheme.
def arctan_taylor(x, terms=9):
"""
Compute arctan for small x via Taylor polynomials.
Uses a fixed number of terms. The default of 9 should give good results for
abs(x) < 0.1. Results will become poorer as abs(x) increases, becoming
unusable as abs(x) approaches 1.0 (the radius of convergence of the
series).
"""
# Uses Horner's method for evaluation.
t = 0.0
for n in range(2*terms-1, 0, -2):
t = 1.0/n - x*x*t
return x * t
The above code gives good results for small x (say smaller than 0.1 in absolute value), but the accuracy drops off as x becomes larger, and for abs(x) > 1.0, the series never converges, no matter how many terms (or how much extra precision) we throw at it. So we need a better way to compute for larger x. One solution is to use argument reduction, via the identity arctan(x) = 2 * arctan(x / (1 + sqrt(1 + x^2))). This gives the following code, which builds on arctan_taylor to give reasonable results for a wide range of x (but beware possible overflow and underflow when computing x*x).
import math
def arctan_taylor_with_reduction(x, terms=9, threshold=0.1):
"""
Compute arctan via argument reduction and Taylor series.
Applies reduction steps until x is below `threshold`,
then uses Taylor series.
"""
reductions = 0
while abs(x) > threshold:
x = x / (1 + math.sqrt(1 + x*x))
reductions += 1
return arctan_taylor(x, terms=terms) * 2**reductions
Alternatively, given an existing implementation for tan, you could simply find a solution y to the equation tan(y) = x using traditional root-finding methods. Since arctan is already naturally bounded to lie in the interval (-pi/2, pi/2), bisection search works well:
def arctan_from_tan(x, tolerance=1e-15):
"""
Compute arctan as the inverse of tan, via bisection search. This assumes
that you already have a high quality tan function.
"""
low, high = -0.5 * math.pi, 0.5 * math.pi
while high - low > tolerance:
mid = 0.5 * (low + high)
if math.tan(mid) < x:
low = mid
else:
high = mid
return 0.5 * (low + high)
Finally, just for fun, here's a CORDIC-like implementation, which is really more appropriate for a low-level implementation than for Python. The idea here is that you precompute, once and for all, a table of arctan values for 1, 1/2, 1/4, etc., and then use those to compute general arctan values, essentially by computing successive approximations to the true angle. The remarkable part is that, after the precomputation step, the arctan computation involves only additions, subtractions, and multiplications by by powers of 2. (Of course, those multiplications aren't any more efficient than any other multiplication at the level of Python, but closer to the hardware, this could potentially make a big difference.)
cordic_table_size = 60
cordic_table = [(2**-i, math.atan(2**-i))
for i in range(cordic_table_size)]
def arctan_cordic(y, x=1.0):
"""
Compute arctan(y/x), assuming x positive, via CORDIC-like method.
"""
r = 0.0
for t, a in cordic_table:
if y < 0:
r, x, y = r - a, x - t*y, y + t*x
else:
r, x, y = r + a, x + t*y, y - t*x
return r
Each of the above methods has its strengths and weaknesses, and all of the above code can be improved in a myriad of ways. I encourage you to experiment and explore.
To wrap it all up, here are the results of calling the above functions on a small number of not-very-carefully-chosen test values, comparing with the output of the standard library math.atan function:
test_values = [2.314, 0.0123, -0.56, 168.9]
for value in test_values:
print("{:20.15g} {:20.15g} {:20.15g} {:20.15g}".format(
math.atan(value),
arctan_taylor_with_reduction(value),
arctan_from_tan(value),
arctan_cordic(value),
))
Output on my machine:
1.16288340166519 1.16288340166519 1.16288340166519 1.16288340166519
0.0122993797673 0.0122993797673 0.0122993797673002 0.0122993797672999
-0.510488321916776 -0.510488321916776 -0.510488321916776 -0.510488321916776
1.56487573286064 1.56487573286064 1.56487573286064 1.56487573286064
The simplest way to do any inverse function is to use binary search.
definitions
let assume function
x = g(y)
And we want to code its inverse:
y = f(x) = f(g(y))
x = <x0,x1>
y = <y0,y1>
bin search on floats
You can do it on integer math accessing mantissa bits like in here:
Any Faster RMS Value Calculation in C?
but if you do not know the exponent of the result prior to computation then you need to use floats for bin search too.
so the idea behind binary search is to change mantissa of y from y1 to y0 bit by bit from MSB to LSB. Then call direct function g(y) and if the result cross x revert the last bit change.
In case of using floats you can use variable that will hold approximate value of the mantissa bit targeted instead of integer bit access. That will eliminate unknown exponent problem. So at the beginning set y = y0 and actual bit to MSB value so b=(y1-y0)/2. After each iteration halve it and do as many iterations as you got mantissa bits n... This way you obtain result in n iterations within (y1-y0)/2^n accuracy.
If your inverse function is not monotonic break it into monotonic intervals and handle each as separate binary search.
The function increasing/decreasing just determine the crossing condition direction (use of < or >).
C++ acos example
so y = acos(x) is defined on x = <-1,+1> , y = <0,M_PI> and decreasing so:
double f64_acos(double x)
{
const int n=52; // mantisa bits
double y,y0,b;
int i;
// handle domain error
if (x<-1.0) return 0;
if (x>+1.0) return 0;
// x = <-1,+1> , y = <0,M_PI> , decreasing
for (y= 0.0,b=0.5*M_PI,i=0;i<n;i++,b*=0.5) // y is min, b is half of max and halving each iteration
{
y0=y; // remember original y
y+=b; // try set "bit"
if (cos(y)<x) y=y0; // if result cross x return to original y decreasing is < and increasing is >
}
return y;
}
I tested it like this:
double x0,x1,y;
for (x0=0.0;x0<M_PI;x0+=M_PI*0.01) // cycle all angle range <0,M_PI>
{
y=cos(x0); // direct function (from math.h)
x1=f64_acos(y); // my inverse function
if (fabs(x1-x0)>1e-9) // check result and output to log if error
Form1->mm_log->Lines->Add(AnsiString().sprintf("acos(%8.3lf) = %8.3lf != %8.3lf",y,x0,x1));
}
Without any difference found... so the implementation is working correctly. Of coarse binary search on 52 bit mantissa is usually slower then polynomial approximation ... on the other hand the implementation is so simple ...
[Notes]
If you do not want to take care of the monotonic intervals you can try
approximation search
As you are dealing with goniometric functions you need to handle singularities to avoid NaN or division by zero etc ...
If you're interested here more bin search examples (mostly on integers)
Power by squaring for negative exponents it contains
Related
I am trying to numerically compute a double integral.
The issue is that (I think) I need a mix of symbolic integration and numerical integration.
The integral looks something like this:
I cannot use numpy.integrate because it is not just a double integral because of the power (1/a) in the middle.
I cannot get a number for the innermost integral (to then raise to the power) because it ends up being a function that depends on x which I would then need to integrate.
I tried with symbolic calculus, using a nested sym.integrate like here
sym.integrate((sym.integrate(sym.exp(-(w**2)/(2*sigmaw)-alpha*((x-w)**2)/(2*sigma)),(w,-sym.oo, sym.oo)))**(1/alpha),(x,-sym.oo, sym.oo))
however, it just spits back the expression itself and no number.
I think I would need to get a symbolic expression for the inner integral to use as a function for numerical integration.
Is it even possible?
If not in python, with another language like R?
Any experience with things of this sort?
I worked with Maxima (https://maxima.sourceforge.io) since OP seems to be saying the exact system used isn't too important.
The integrand is just a product of Gaussian bumps, so its integral over the real line is not too hard. Maxima doesn't have the strongest integrator in the world, but anyway it seems to handle this problem okay.
Start by assuming all the parameters are positive; if not specified, Maxima will ask for the sign during the calculation.
(%i2) assume (alpha > 0, sigmaw > 0, sigma > 0);
(%o2) [alpha > 0, sigmaw > 0, sigma > 0]
Define the inner integrand.
(%i3) I: exp(-(w**2)/(2*sigmaw)-alpha*((x-w)**2)/(2*sigma));
2 2
alpha (x - w) w
(- --------------) - --------
2 sigma 2 sigmaw
(%o3) %e
Compute the inner integral.
(%i4) I1: integrate (I, w, minf, inf);
(%o4) (sqrt(2) sqrt(%pi) sqrt(sigma) sqrt(sigmaw)
2
alpha x
- ------------------------
2 alpha sigmaw + 2 sigma
%e )/sqrt(alpha sigmaw + sigma)
The pretty-printer (ASCII art) display is hard to read here, maybe this 1-d representation makes more sense. grind produces the 1-d display.
(%i5) grind(%);
(sqrt(2)*sqrt(%pi)*sqrt(sigma)*sqrt(sigmaw)
*%e^-((alpha*x^2)/(2*alpha*sigmaw+2*sigma)))
/sqrt(alpha*sigmaw+sigma)$
(%o5) done
Define the outer integrand.
(%i7) I2: I1^(1/alpha);
1 1 1 1
------- ------- ------- -------
2 alpha 2 alpha 2 alpha 2 alpha
(%o7) (2 %pi sigma sigmaw
2
x 1
- ------------------------ -------
2 alpha sigmaw + 2 sigma 2 alpha
%e )/(alpha sigmaw + sigma)
Compute the outer integral. The final result is named foo here.
(%i9) foo: integrate (I2, x, minf, inf);
1 1 1 1
------- + 1/2 ------- ------- -------
2 alpha 2 alpha 2 alpha 2 alpha
(%o9) (%pi 2 sigma sigmaw
1
-------
2 alpha
sqrt(2 alpha sigmaw + 2 sigma))/(alpha sigmaw + sigma)
Evaluate the outer integral for specific values of the parameters.
(%i10) ev (foo, alpha = 3, sigma = 3/7, sigmaw = 7/4);
1/6 1/6 1/6 1/3 2/3
2 3 7 159 %pi
(%o10) ----------------------------
sqrt(14)
(%i11) float(%);
(%o11) 5.790416728790489
Compute a numerical approximation. Note quad_qagi is suitable for infinite intervals.
(%i12) ev (quad_qagi (lambda([x], quad_qagi (I, w, minf, inf)[1]^(1/alpha)), x, minf, inf),
alpha = 3, sigma = 3/7, sigmaw = 7/4);
(%o12) [5.790416728790598, 7.216782674725913E-9, 270, 0]
Looks like that supports the symbolic result.
(%i13) first(%) - %o11;
(%o13) 1.092459456231154E-13
The outer integral again, in 1-d display which might be useful for copying into another program:
(%i14) grind(foo);
(%pi^(1/(2*alpha)+1/2)*2^(1/(2*alpha))*sigma^(1/(2*alpha))
*sigmaw^(1/(2*alpha))
*sqrt(2*alpha*sigmaw+2*sigma))
/(alpha*sigmaw+sigma)^(1/(2*alpha))$
(%o14) done
I recommend pretty strongly to try to get to a symbolic result if possible; numerical integration is often tricky. In the example given, if it turned out that you could only do the inner integral but not the outer one, that would still be a pretty big win. You could plug the symbolic solution for the inner integral into a numerical approximation for the outer one.
this doesn't answer your question but it will surely help you, as other have already pointed out other useful tools.
for the integration at hand, you don't really need to do symbolic integration.
numerical integration is simply summing on a defined finite grid, and integrating over w is simply summing over the w axis, same as x.
the main problem is how to choose the integration grid, since it cannot be infinite, for gaussians I'd say at least 10 times their sigma for as low error as you can get, as for the grid spacing, I'd make it as small as you can wait for it to run.
so for the above integration, this would be equivalent, make sure you don't increase the grid steps until you have a picture of how much memory it will need, or else your pc will hang.
import numpy as np
# define constants
sigmaw = 0.1
sigma = 0.1
alpha = 0.2
# define grid
max_w = 2
min_w = -max_w
min_x = -3
max_x = -min_x
steps_w = 2000 # don't increase this too much or you'll run out of memory
steps_x = 1000 # don't increase this too much or you'll run out of memory
dw = (max_w - min_w) / steps_w
dx = (max_x - min_x) / steps_x
x_vec = np.linspace(min_x, max_x, steps_x)
w_vec = np.linspace(min_w, max_w, steps_w)
x, w = np.meshgrid(x_vec, w_vec, sparse=True)
# do integration
inner_term = np.exp(-(w ** 2) / (2 * sigmaw) - alpha * ((x - w) ** 2) / (2 * sigma))
inner_integral = np.sum(inner_term, axis=0) * dw
del inner_term # to free some memory
inner_integral_powered = inner_integral ** (1 / alpha)
del inner_integral # to free some memory
outer_integral = np.sum(inner_integral_powered) * dx
print(outer_integral)
Numerical integration works by sampling the integrand at some values of the argument. In particular, the Newton-Cotes formulas sample uniformly, while different flavors of Gaussian integration sample irregularly.
So in your case, the integrator will require an evaluation of the inner integral for various values of x to integrate on x, implying each time a numerical integration on w with known x.
Note that as your domain is unbounded, you will have to use a change of variable to make it finite.
If the inner integral has an analytical expression, you can of course use it and integrate numerically on x.
I'm trying to numerically solve the equation x=a*sin(x), where a is some constant, in python. I already tried first solving the equation symbolically, but it seems this particular shape of expression isn't implemented in sympy. I also tried using sympy.nsolve(), but it only gives me the first solution it encounters.
My plan looks something like this:
x=0
a=1
rje=[]
while(x<a):
if (x-numpy.sin(x))<=error_sin:
rje.append(x)
x+=increment
print(rje)
I don't want to waste time or risk missing solutions, so I want to know how to find out how precise numpy's sinus is on my device (that would become error_sin).
edit: I tried making both error_sin and increment equal to the machine epsilon of my device but it a) takes to much time, and b) sin(x) is less precise that x and so I get a lot of non-solutions (or rather repeated solutions because sin(x) grows much slower than x). Hence the question.
edit2: Could you please just help me answer the question about precision of numpy.sin(x)? I provided information about the purpose purely for context.
The answer
np.sin will in general be as precise as possible, given the precision of the double (ie 64-bit float) variables in which the input, output, and intermediate values are stored. You can get a reasonable measure of the precision of np.sin by comparing it to the arbitrary precision version of sin from mpmath:
import matplotlib.pyplot as plt
import mpmath
from mpmath import mp
# set mpmath to an extremely high precision
mp.dps = 100
x = np.linspace(-np.pi, np.pi, num=int(1e3))
# numpy sine values
y = np.sin(x)
# extremely high precision sine values
realy = np.array([mpmath.sin(a) for a in x])
# the end results are arrays of arbitrary precision mpf values (ie abserr.dtype=='O')
diff = realy - y
abserr = np.abs(diff)
relerr = np.abs(diff/realy)
plt.plot(x, abserr, lw=.5, label='Absolute error')
plt.plot(x, relerr, lw=.5, label='Relative error')
plt.axhline(2e-16, c='k', ls='--', lw=.5, label=r'$2 \cdot 10^{-16}$')
plt.yscale('log')
plt.xlim(-np.pi, np.pi)
plt.ylim(1e-20, 1e-15)
plt.xlabel('x')
plt.ylabel('Error in np.sin(x)')
plt.legend()
Output:
Thus, it is reasonable to say that both the relative and absolute errors of np.sin have an upper bound of 2e-16.
A better answer
There's an excellent chance that if you make increment small enough for your approach to be accurate, your algorithm will be too slow for practical use. The standard equation solving approaches won't work for you, since you don't have a standard function. Instead, you have an implicit, multi-valued function. Here's a stab at a general purpose approach for getting all solutions to this kind of equation:
import matplotlib.pyplot as plt
import numpy as np
import scipy.optimize as spo
eps = 1e-4
def func(x, a):
return a*np.sin(x) - x
def uniqueflt(arr):
b = arr.copy()
b.sort()
d = np.append(True, np.diff(b))
return b[d>eps]
initial_guess = np.arange(-9, 9) + eps
# uniqueflt removes any repeated roots
roots = uniqueflt(spo.fsolve(func, initial_guess, args=(10,)))
# roots is an array with the 7 unique roots of 10*np.sin(x) - x == 0:
# array([-8.42320394e+00, -7.06817437e+00, -2.85234190e+00, -8.13413225e-09,
# 2.85234189e+00, 7.06817436e+00, 8.42320394e+00])
x = np.linspace(-20, 20, num=int(1e3))
plt.plot(x, x, label=r'$y = x$')
plt.plot(x, 10*np.sin(x), label=r'$y = 10 \cdot sin(x)$')
plt.plot(roots, 10*np.sin(roots), '.', c='k', ms=7, label='Solutions')
plt.ylim(-10.5, 20)
plt.gca().set_aspect('equal', adjustable='box')
plt.legend()
Output:
You'll have to tweak the initial_guess depending on your value of a. initial_guess has to be at least as large as the actual number of solutions.
The accuracy of the sine function is not so relevant here, you'd better perform the study of the equation.
If you write it in the form sin x / x = sinc x = 1 / a, you immediately see that the number of solutions is the number of intersections of the cardinal sine with an horizontal. This number depends on the ordinates of the extrema of the latter.
The extrema are found where x cos x - sin x = 0 or x = tan x, and the corresponding values are cos x. This is again a transcendental equation, but it is parameterless and you can solve it once for all. Also note that for increasing values of x, the solutions get closer and closer to (k+1/2)π.
Now for a given value of 1 / a, you can find all the extrema below and above and this will give you starting intervals where to look for the roots. The secant method will be handy.
A simple way to estimate the accuracy of sin() AND cos() for a given argument x would be:
eps_trig = np.abs(1 - (np.sin(x)**2 + np.cos(x)**2)) / 2
You may want to drop last 2 just to be on the "safe side" (well, there are values of x for which this approximation does not hold very well, in particular for x close to -90 deg). I would suggest to test at around x=pi/4
Explanation:
The basic idea behind this approach is as follows... Let's say our sin(x) and cos(x) deviates from exact values by a single "error value" eps. That is, exact_sin(x) = sin(x) + eps (same for cos(x)). Also, let's call delta to be the measured deviation from the Pythagorean trigonometric identity:
delta = 1 - sin(x)**2 - cos(x)**2
For exact functions, delta should be zero:
1 - exact_sin(x)**2 - exact_cos(x)**2 == 0
or, going to inexact functions:
1 - (sin(x) + eps)**2 - (cos(x) + eps)**2 == 0 =>
1 - sin(x)**2 - cos(x)**2 = delta = 2*eps*(sin(x) + cos(x)) + 2*eps**2
Neglecting last term 2*eps**2 (assume small errors):
2*eps*(sin(x)+cos(x)) = 1 - sin(x)**2 - cos(x)**2
If we choose x such that sin(x)+cos(x) hovers around 1 (or, somewhere in the range 0.5-2), we can roughly estimate that eps = |1 - sin(x)**2 - cos(x)**2|/2.
To the precision you already got good answers. To the task itself, you can be faster by investing some calculus.
First, from the bounds of the sine you know that any solution must be in the interval [-abs(a),abs(a)]. If abs(a)\le 1 then the only root in [-1,1] is x=0
Apart from the interval containing zero, you also know that there is exactly one root in any of the intervals between the roots of cos(x)=1/a which are the extrema of a*sin(x)-x. Set phi=arccos(1/a) in [0,pi], then these roots are -phi+2*k*pi and phi+2*k*pi.
The interval for k=0 might contain 3 roots if 1<a<0.5*pi. For the positive root one knows x/a=sin(x)>x-x^3/6 so that x^2>6-6/a.
And lastly, the problem is symmetric, if x is a root, so is -x so all you have to do is find the positive roots.
So to compute the roots,
Start the root list with the root 0.
in the case abs(a)<=1, there are no further roots, return. One could also use -pi/2<=a<=1.
in the case 1<a<pi/2, apply the chosen bracketing method to the interval [sqrt(6-6/a), pi/2], add the root to the list, and return.
In the remaining cases where abs(a)>=0.5*pi:
Compute phi=arccos(1/a).
Then for any positive integer k apply the bracketing method to the intervals [2*(k-1)*pi+phi,2*k*pi-phi] and [2*k*pi-phi,2*k*pi-phi so that (k-0.5)*pi < abs(a) [(k-0.5)*pi, (k+0.5)*pi] as long as the lower interval boundary is smaller than abs(a) and the function has a sign change over the interval.
Add the root found to the list. Return with the list after the loop ends.
let a=10;
function f(x) { return x - a * Math.sin(x); }
findRoots();
//-------------------------------------------------
function findRoots() {
log.innerHTML = `<p>roots for parameter a=${a}`;
rootList.innerHTML = "<tr><th>root <i>x</i></th><th><i>x-a*sin(x)</i></th><th>numSteps</th></tr>";
rootList.innerHTML += "<tr><td>0.0<td>0.0<td>0</tr>";
if( Math.abs(a)<=1) return;
if( (1.0<a) && (a < 0.5*Math.PI) ) {
illinois(Math.sqrt(6-6/a), 0.5*Math.PI);
return;
}
const phi = Math.acos(1.0/a);
log.innerHTML += `phi=${phi}<br>`;
let right = 2*Math.PI-phi;
for (let k=1; right<Math.abs(a); k++) {
let left = right;
right = (k+2)*Math.PI + ((0==k%2)?(-phi):(phi-Math.PI));
illinois(left, right);
}
}
function illinois(a, b) {
log.innerHTML += `<p>regula falsi variant illinois called for interval [a,b]=[${a}, ${b}]`;
let fa = f(a);
let fb = f(b);
let numSteps=2;
log.innerHTML += ` values f(a)=${fa}, f(b)=${fb}</p>`;
if (fa*fb > 0) return;
if (Math.abs(fa) < Math.abs(fb)) { var h=a; a=b; b=h; h=fa; fa=fb; fb=h;}
while(Math.abs(b-a) > 1e-15*Math.abs(b)) {
let c = b - fb*(b-a)/(fb-fa);
let fc = f(c); numSteps++;
log.innerHTML += `step ${numSteps}: midpoint c=${c}, f(c)=${fc}<br>`;
if ( fa*fc < 0 ) {
fa *= 0.5;
} else {
a = b; fa = fb;
}
b = c; fb = fc;
}
rootList.innerHTML += `<tr><td>${b}<td>${fb}<td>${numSteps}</tr>`;
}
aInput.addEventListener('change', () => {
let a_new = Number.parseFloat(aInput.value);
if( isNaN(a_new) ) {
alert('Not a number '+aInput.value);
} else if(a!=a_new) {
a = a_new;
findRoots();
}
});
<p>Factor <i>a</i>: <input id="aInput" value="10" /></p>
<h3>Root list</h3>
<table id="rootList" border = 1>
</table>
<h3>Computation log</h3>
<div id="log"/>
The solution should be precise up to machine epsilon
>>> from numpy import sin as sin_np
>>> from math import sin as sin_math
>>> x = 0.0
>>> sin_np(x) - x
0.0
>>> sin_math(x) - x
0.0
>>>
You could consider using scipy.optimize for this problem:
>>> from scipy.optimize import minimize
>>> from math import sin
>>> a = 1.0
Then define your objective as so:
>>> def obj(x):
... return abs(x - a*sin(x))
...
And you can go ahead and solve this problem numerically by:
>>> sol = minimize(obj, 0.0)
>>> sol
fun: array([ 0.])
hess_inv: array([[1]])
jac: array([ 0.])
message: 'Optimization terminated successfully.'
nfev: 3
nit: 0
njev: 1
status: 0
success: True
x: array([ 0.])
Now lets try with a new value of a
>>> a = .5
>>> sol = minimize(obj, 0.0)
>>> sol
fun: array([ 0.])
hess_inv: array([[1]])
jac: array([ 0.5])
message: 'Desired error not necessarily achieved due to precision loss.'
nfev: 315
nit: 0
njev: 101
status: 2
success: False
x: array([ 0.])
>>>
In case you want to find a non-trivial solution to this problem, you need to change x0 iteratively to values greater than zero and also lesser than. Also, manage the bounds of x in minimise by setting bounds in scipy.optimize.minimize, you would be able to walk from -infty to +infty ( or very large numbers ).
I have been trying to solve integration with riemann sum. My function has 3 arguments a,b,d so a is lower limit b is higher limit and d is the part where a +(n-1)*d < b. This is my code so far but. My output is 28.652667999999572 what I should get is 28.666650000000388. Also if the input b is lower than a it has to calculate but I have solved that problem already.
def integral(a, b, d):
if a > b:
a,b = b,a
delta_x = float((b-a)/1000)
j = abs((b-a)/delta_x)
i = int(j)
n = s = 0
x = a
while n < i:
delta_A = (x**2+3*x+4) * delta_x
x += delta_x
s += delta_A
n += 1
return abs(s)
print(integral(1,3,0.01))
There is no fault here, neither with the algorithm nor with your code (or python). The Riemann sum is an approximation of the integral and per se not "exact". You approximate the area of a (small) stripe of width dx, say between x and x+dx, and f(x) with the area of an rectangle of the same width and the height of f(x) as it's left upper corner. If the function changes it's value when you go from x to x+dx then the area of the rectangle deviates from the true integral.
As you have noticed, you can make the approximation closer by making thinner and thinner slices, at the cost of more computational effort and time.
In your example, the function is f(x) = x^2 + 3*x + 4, and it's exact integral over x in [1.0,3.0) is 28 2/3 or 28.66666...
The approximation by rectangles is a crude one, you cannot change that. But what you could change is the time it takes for your code to evaluate, say, 10^8 steps instead of 10^3. Look at this code:
def riemann(a, b, dx):
if a > b:
a,b = b,a
# dx = (b-a)/n
n = int((b - a) / dx)
s = 0.0
x = a
for i in xrange(n):
f_i = (x + 3.0) * x + 4.0
s += f_i
x += dx
return s * dx
Here, I've used 3 tricks for speedup, and one for greater precision. First, if you write a loop and you know the number of repetions in advance then use a for-loop instead of a while-loop. It's faster. (BTW, loop variables conventionally are i, j, k ... whereas a limit or final value is n). Secondly, using xrange instead of range is faster for users of python 2.x. Thirdly, factorize polynoms when calculating them often. You should see from the code what I mean here. This way, the result is numerically stable. Last trick: operations within the loop which do not depend on the loop variable can be extracted and applied after the loop has ended. Here, the final multiplication with dx.
I'm trying to calculate (3e28 choose 2e28)/2^(3e28). I tried scipy.misc.comb to calculate 3e28 choose 2e28 but it gave me inf. When I calculate 2^(3e28), it raised OverflowError: (34, 'Result too large'). How can I compute or estimate (3e28 choose 2e28)/2^(3e28)?
Use Stirling's approximation (which is very accurate in the 1e10+ range), combined with logarithms:
(3e28 choose 2e28) / 2^(3e28) = 3e28! / [(3e28 - 2e28)! * 2e28!] / 2^(3e28)
= e^ [log (3e28!) - log((3e28-2e28)!) - log(2e28!) - 3e28 * log(2)]
and from there apply Stirling's approximation:
log n! ~= log(sqrt(2*pi*n)) + n*log(n) - n
and you'll get your answer.
Here's an example of how accurate this approximation is:
>>> import math
>>> math.log(math.factorial(100))
363.73937555556347
>>> math.log((2*math.pi*100)**.5) + 100*math.log(100) - 100
363.7385422250079
For 100!, it's off by less than 0.01% in log-space.
You can compute this ratio with the normal approximation to the binomial for large n. When n is large, k has to be relatively close to n/2 for (n choose k) / 2^n to not be negligible.
Code
Here's some code that will compute this:
def n_choose_k_over_2_pow_n(n, k):
# compute the mean and standard deviation of the normal
# approximation
mu = n / 2.
sigma = np.sqrt(n) * 1/4.
# now transform to a standard normal variable
z = (k - mu) / sigma
return 1/np.sqrt(2*np.pi) * np.exp(-1/2. * z**2)
So that:
>>> n_choose_k_over_2_pow_n(3e28, 2e28)
0.0
>>> n_choose_k_over_2_pow_n(3e28, 1.5e28)
0.3989422804014327
As you can see, the computation underflows. A solution is to compute the log of the answer, which we can do with this code:
def log_n_choose_k_over_2_pow_n(n, k):
# compute the mean and standard deviation of the normal
# approximation
mu = n / 2.
sigma = np.sqrt(n) * 1/4.
# now transform to a standard normal variable
z = (k - mu) / sigma
# return the log of the answer
return -1./2 * (np.log(2 * np.pi) + z**2)
Another quick check:
>>> log_n_choose_k_over_2_pow_n(3e28, 2e28)
-6.6666666666666638e+27
>>> log_n_choose_k_over_2_pow_n(3e28, 1.5e28)
-0.91893853320467267
If we exponentiate these, we'll get our previous answers.
Explanation
We can do this by an appeal to results from statistics. The binomial distribution is given by:
P(K = k) = (n choose k) p^k * p^(n-k)
For large n, this is well-approximated by the normal distribution with mean n*p and variance n*p*(1-p).
Set p to be 1/2. Then we have:
P(K = k) = (n choose k) (1/2)^k * (1/2)^(n-k)
= (n choose k) (1/2)^n
= (n choose k) / (2^n)
Which is precisely the form of your ratio. Therefore, after a transformation to a standard normal variable with mean n/2 and variance n/4, we can compute your ratio by a simple evaluation of the standard normal distribution pdf.
The following uses log2comb from my answer here:
from math import log
from scipy.special import gammaln
def log2comb(n, k):
return (gammaln(n+1) - gammaln(n-k+1) - gammaln(k+1)) / log(2)
log2p = log2comb(3e28, 2e28) - 3e28
print "log2p =", log2p
which prints
log2p = -2.45112497837e+27
So the base-2 logarithm of your number is about -2.45e27. If you try to compute 2**log2p, you get 0. That is, the number is smaller than the smallest positive number representable with standard 64 bit floating point numbers.
There are python libraries that allow you to do arbitrary precision arithmetic. For example mpmath as used in SymPy.
You will have to rewrite your code to use the library functions though.
http://docs.sympy.org/latest/modules/mpmath/basics.html?highlight=precision
Edit: I just noticed the size of the numbers you are dealing with - much too large for my suggestion.
The following was ported from the pseudo-code from the Wikipedia article on Newton's method:
#! /usr/bin/env python3
# https://en.wikipedia.org/wiki/Newton's_method
import sys
x0 = 1
f = lambda x: x ** 2 - 2
fprime = lambda x: 2 * x
tolerance = 1e-10
epsilon = sys.float_info.epsilon
maxIterations = 20
for i in range(maxIterations):
denominator = fprime(x0)
if abs(denominator) < epsilon:
print('WARNING: Denominator is too small')
break
newtonX = x0 - f(x0) / denominator
if abs(newtonX - x0) < tolerance:
print('The root is', newtonX)
break
x0 = newtonX
else:
print('WARNING: Not able to find solution within the desired tolerance of', tolerance)
print('The last computed approximate root was', newtonX)
Question
Is there an automated way to calculate some form of fprime given some form of f in Python 3.x?
A common way of approximating the derivative of f at x is using a finite difference:
f'(x) = (f(x+h) - f(x))/h Forward difference
f'(x) = (f(x+h) - f(x-h))/2h Symmetric
The best choice of h depends on x and f: mathematically the difference approaches the derivative as h tends to 0, but the method suffers from loss of accuracy due to catastrophic cancellation if h is too small. Also x+h should be distinct from x. Something like h = x*1e-15 might be appropriate for your application. See also implementing the derivative in C/C++.
You can avoid approximating f' by using the secant method. It doesn't converge as fast as Newton's, but it's computationally cheaper and you avoid the problem of having to calculate the derivative.
You can approximate fprime any number of ways. One of the simplest would be something like:
lambda fprime x,dx=0.1: (f(x+dx) - f(x-dx))/(2*dx)
the idea here is to sample f around the point x. The sampling region (determined by dx) should be small enough that the variation in f over that region is approximately linear. The algorithm that I've used is known as the midpoint method. You could get more accurate by using higher order polynomial fits for most functions, but that would be more expensive to calculate.
Of course, you'll always be more accurate and efficient if you know the analytical derivative.
Answer
Define the functions formula and derivative as the following directly after your import.
def formula(*array):
calculate = lambda x: sum(c * x ** p for p, c in enumerate(array))
calculate.coefficients = array
return calculate
def derivative(function):
return (p * c for p, c in enumerate(function.coefficients[1:], 1))
Redefine f using formula by plugging in the function's coefficients in order of increasing power.
f = formula(-2, 0, 1)
Redefine fprime so that it is automatically created using functions derivative and formula.
fprime = formula(*derivative(f))
That should solve your requirement to automatically calculate fprime from f in Python 3.x.
Summary
This is the final solution that produces the original answer while automatically calculating fprime.
#! /usr/bin/env python3
# https://en.wikipedia.org/wiki/Newton's_method
import sys
def formula(*array):
calculate = lambda x: sum(c * x ** p for p, c in enumerate(array))
calculate.coefficients = array
return calculate
def derivative(function):
return (p * c for p, c in enumerate(function.coefficients[1:], 1))
x0 = 1
f = formula(-2, 0, 1)
fprime = formula(*derivative(f))
tolerance = 1e-10
epsilon = sys.float_info.epsilon
maxIterations = 20
for i in range(maxIterations):
denominator = fprime(x0)
if abs(denominator) < epsilon:
print('WARNING: Denominator is too small')
break
newtonX = x0 - f(x0) / denominator
if abs(newtonX - x0) < tolerance:
print('The root is', newtonX)
break
x0 = newtonX
else:
print('WARNING: Not able to find solution within the desired tolerance of', tolerance)
print('The last computed approximate root was', newtonX)