I am trying to code a Newton's Method with back stepping code that I wrote in Matlab to Python, but am having some trouble with the Python syntax. Matlab takes about 5 iterations, but my Python code is looping up to the max iteration of 1000 and having a domain error as the back stepping mechanism does not work (tries to calculate a negative log). I have not used Python in a while so I am most likely confusing some syntax of some sort.
Here is the Matlab code that works correctly:
x = 10; %defines x
f = #(x) log(x); %defines objective function
df = #(x) 1/x; %defines first derivative
tol = .00001; %defines our tolerance level
maxit = 1000; %defines maximum iteration steps
maxsteps = 200; %defines maximum backsteps
for i=1:maxit %starts loop
fval = f(x); %value of function at f(x)
fjac = df(x); %value of jacobian at f(x)
fnorm = norm(fval); %calculates norm value at fval
if fnorm<tol, return, end %if fnorm less than tol, end
x
d = -(fjac\fval); %forms second part of iteration rule
d
fnormold = inf; %sets arbitrary fnormold
for backstep=1:maxsteps
fvalnew = f(x+d); %calculates f(x+d)
fnormnew = norm(fvalnew); %calculates norm of fvalnew
if fnormnew<fnorm, break, end %implements 1st backstepping rule
if fnormold<fnormnew, d=2*d; break, end %implements 2nd backstepping rule
fnormold = fnormnew; %updates fnormold
d=d/2;
end
x=x+d;
end
disp(x)
Here is the Python code:
from math import log
x = 10
def f(x):
f = x* log(x)
return f
def df(x):
df = 1/x
return df
tol = .00001
maxit = 1000
maxsteps = 200
maxsteps = 200
for i in range(1, maxit):
fval = f(x)
fjac = df(x)
fnorm = abs(fval)
if fnorm < tol:
print x
d = -(fjac/fval)
fnormold = float('Inf')
for backstep in range(1, maxsteps):
fvalnew = f(x+d)
fnormnew = abs(fvalnew)
if fnormnew < fnorm:
break
if fnormold < fnormnew:
d= 2*d
break
fnormold = fnormnew
d = d/2
x = x+d
print x
1/x in df can be 0 in most cases since in python 2.x divison over integers returns integer
range based for is one index too less
Related
I would like to find an approximate value for the number pi = 3.14.. by using the Newton method. In order to use it also for some other purpose and thus other function than sin(x), the aim is to implement a generic function that will be passed over as an argument. I have an issue in passing a function as an argument into an other function. I also tried lambda in different variations. The code I am showing below produces the error message: IndexError: list index out of range. I will appreciate your help in solving this issue and eventually make any suggestion in the code which may not be correct. Thanks.
from sympy import *
import numpy as np
import math
x = Symbol('x')
# find the derivative of f
def deriv(f,x):
h = 1e-5
return (lambda x: (f(x+h)-f(x))/h)
def newton(x0,f,err):
A = [x0]
n = 1
while abs(A[n]-A[n-1])<=err:
if n == 1:
y = lambda x0: (math.f(x0))
b = x0-y(x0)/(deriv(y,x0))
A.append(b)
n += 1
else:
k = len(A)
xk = A[k]
y = lambda xk: (math.f(xk))
b = newton(A[k],y,err)-y(newton(A[k],y,err))/deriv(y,k)
A.append(b)
n += 1
return A, A[-1]
print(newton(3,math.sin(3),0.000001))
I don't know why you use sympy because I made it without Symbol
At the beginning you have to calculate second value and append it to list A and later you can calculate abs(A[n]-A[n-1]) (or the same without n: abs(A[-1] - A[-2])) because it needs two values from this list.
Other problem is that it has to check > instead of <=.
If you want to send function sin(x) then you have to use math.sin without () and arguments.
If you want to send function sin(3*x) then you would have to use lambda x: math.sin(3*x)
import math
def deriv(f, x, h=1e-5):
return (f(x+h) - f(x)) / h
def newton(x0, f, err):
A = [x0]
x = A[-1] # get last value
b = x - (f(x) / deriv(f, x)) # calculate new value
A.append(b) # add to list
while abs(A[-1] - A[-2]) > err: # it has to be `>` instead of `<=`
x = A[-1] # get last value
b = x - (f(x) / deriv(f, x)) # calculate new value
A.append(b) # add to list
return A, A[-1]
# sin(x)
print(newton(3, math.sin, 0.000001)) # it needs function's name without `()`
# sin(3*x)
print(newton(3, lambda x:math.sin(3*x), 0.000001))
# sin(3*x) # the same without `lambda`
def function(x):
return math.sin(3*x)
print(newton(3, function, 0.000001))
Result:
([3, 3.1425464414785056, 3.1415926532960112, 3.141592653589793], 3.141592653589793)
([3, 3.150770863559604, 3.1415903295877707, 3.1415926535897936, 3.141592653589793], 3.141592653589793)
EDIT:
You may write loop in newton in different way and it will need <=
def newton(x0, f, err):
A = [x0]
while True:
x = A[-1] # get last value
b = x - (f(x) / deriv(f, x)) # calculate new value
A.append(b) # add to list
if abs(A[-1] - A[-2]) <= err:
break
return A, A[-1]
def f(x):
f='exp(x)-x-2'
y=eval(f)
print(y)
return y
def bissection(f,f_line,f_2lines,a,b,epsilon1,epsilon2):
x=a
result_a=f(a)
x=b
result_b=f(b)
if (f.evalf(a)*f.evalf(b)>=0):
print("Interval [a,b] does not contain a zero ")
exit()
zeta=min(epsilon1,epsilon2)/10
x=a
while(f_line(x)>0):
if(x<b or x>-b):
x=x+zeta
else:
stop
ak=a
bk=b
xk=(ak+bk)/2
k=0
if (f(xk)*f(ak)<0):
ak=ak
bk=xk
if (f(xk)*f(bk)<0):
ak=xk
bk=bk
k=k+1
from sympy import *
import math
x=Symbol('x')
f=exp(x)-x-2
f_line=f.diff(x)
f_2lines=f_line.diff(x)
print("Derivative of f:", f_line)
print("2nd Derivative of f:", f_2lines)
a=int(input('Beginning of interval: '))
b=int(input('End of interval: '))
epsilon1=input('1st tolerance: ')
epsilon2=input('2nd tolerance: ')
bissection(f,f_line,f_2lines,a,b,epsilon1,epsilon2)
This program is an attempt to implement the Bissection Method. I've tried writing two functions:
The first one, f, is supposed to receive the extremes of the interval that may or may not contain a root (a and b) and return the value of the function evaluated in this point.
The second one, bissection, should receive the function, the function's first and second derivatives, the extremes of the interval (a,b) and two tolerances (epsilon1,epsilon2).
What I want to do is pass each value a and b, one at a time, as arguments to the function f, that is supposed to return f(a) and f(b); that is, the values of the function in each of the points a and b.
Then, it should test two conditions:
1) If the function values in the extremes of the intervals have opposite signs. If they don't, the method won't converge for this interval, then the program should terminate.
if(f.evalf(a)*f.evalf(b)>=0)
exit()
2)
while(f_line(x)>0): #while the first derivative of the function evaluated in x is positive
if(x<b or x>-b): #This should test whether x belongs to the interval [a,b]
x=x+zeta #If it does, x should receive x plus zeta
else:
stop
At the end of this loop, my objective was to determine whether the first derivative was strictly positive (I didn't do the negative case yet).
The problem: I'm getting the error
Traceback (most recent call last):
File "bissec.py", line 96, in <module>
bissection(f,f_line,f_2lines,a,b,epsilon1,epsilon2)
File "bissec.py", line 41, in bissection
result_a=f(a)
TypeError: 'Add' object is not callable
How can I properly call the function so that it returns the value of the function (in this case, f(x)=exp(x)-x-2), for every x needed? That is, how can I evaluate f(a) and f(b)?
Ok, so I've figured it out where your program was failing and I've got 4 reasons why.
First of all, and the main topic of your question, if you want to evaluate a function f for a determined x value, let's say a, you need to use f.subs(x, a).evalf(), as it is described in SymPy documentation. You used in 2 different ways: f.evalf(2) and f_line(a); both were wrong and need to be substituted by the correct syntax.
Second, if you want to stop a while loop you should use the keyword break, not "stop", as written in your code.
Third, avoid using the same name for variables and functions. In your f function, you also used f as the name of a variable. In bissection function, you passed f as a parameter and tried to call the f function. That'll fail too. Instead, I've changed the f function to f_calc, and applied the correct syntax of my first point in it.
Fourth, your epsilon1 and epsilon2 inputs were missing a float() conversion. I've added that.
Now, I've also edited your code to use good practices and applied PEP8.
This code should fix this error that you're getting and a few others:
from sympy import *
def func_calc(func, x, val):
"""Evaluate a given function func, whose varible is x, with value val"""
return func.subs(x, val).evalf()
def bissection(x, f, f_line, f_2lines, a, b, epsilon1, epsilon2):
"""Applies the Bissection Method"""
result_a = func_calc(f, x, a)
result_b = func_calc(f, x, b)
if (result_a * result_b >= 0):
print("Interval [a,b] does not contain a zero")
exit()
zeta = min(epsilon1, epsilon2) / 10
x_val = a
while(func_calc(f_line, x, a) > 0):
if(-b < x_val or x_val < b):
x_val = x_val + zeta
else:
break # the keyword you're looking for is break, instead of "stop"
print(x_val)
ak = a
bk = b
xk = (ak + bk) / 2
k = 0
if (func_calc(f, x, xk) * func_calc(f, x, ak) < 0):
ak = ak
bk = xk
if (func_calc(f, x, xk) * func_calc(f, x, bk) < 0):
ak = xk
bk = bk
k = k + 1
def main():
x = Symbol('x')
f = exp(x) - x - 2
f_line = f.diff(x)
f_2lines = f_line.diff(x)
print("Derivative of f:", f_line)
print("2nd Derivative of f:", f_2lines)
a = int(input('Beginning of interval: '))
b = int(input('End of interval: '))
epsilon1 = float(input('1st tolerance: '))
epsilon2 = float(input('2nd tolerance: '))
bissection(x, f, f_line, f_2lines, a, b, epsilon1, epsilon2)
if __name__ == '__main__':
main()
I cannot seem to get an output when I pass numbers to the function. I need to get the computed value and subtract it from the exact. Is there something I am not getting right?
def f1(x):
f1 = np.exp(x)
return f1;
def trapezoid(f,a,b,n):
'''Computes the integral of functions using the trapezoid rule
f = function of x
a = upper limit of the function
b = lower limit of the function
N = number of divisions'''
h = (b-a)/N
xi = np.linspace(a,b,N+1)
fi = f(xi)
s = 0.0
for i in range(1,N):
s = s + fi[i]
s = np.array((h/2)*(fi[0] + fi[N]) + h*s)
print(s)
return s
exactValue = np.full((20),math.exp(1)-1)
a = 0.0;b = 1.0 # integration interval [a,b]
computed = np.empty(20)
E=np.zeros(20)
exact=np.zeros(20)
N=20
def convergence_tests(f, a, b, N):
n = np.zeros(N, 1);
E = np.zeros(N, 1);
Exact = math.exp(1)-1
for i in range(N):
n[i] = 2^i
computed[i] = trapezoid(f, a, b, n[i])
E = abs(Exact - computed)
print(E, computed)
return E, computed
You have defined several functions, but your main program never calls any of them. In fact, your "parent" function convergence_test cannot be called, because it's defined at the bottom of the program.
I suggest that you use incremental programming: write a few lines; test those before you proceed to the next mini-task in your code. In the posting, you've written about 30 lines of active code, without realizing that virtually none of it actually executes. There may well be several other errors in this; you'll likely have a difficult time fixing all of them to get the expected output.
Start small and grow incrementally.
I'm a new learner of python programming. Recently I'm trying to write a "tool" program of "dynamic programming" algorithm. However, the last part of my programe -- a while loop, failed to loop. the code is like
import numpy as np
beta, rho, B, M = 0.5, 0.9, 10, 5
S = range(B + M + 1) # State space = 0,...,B + M
Z = range(B + 1) # Shock space = 0,...,B
def U(c):
"Utility function."
return c**beta
def phi(z):
"Probability mass function, uniform distribution."
return 1.0 / len(Z) if 0 <= z <= B else 0
def Gamma(x):
"The correspondence of feasible actions."
return range(min(x, M) + 1)
def T(v):
"""An implementation of the Bellman operator.
Parameters: v is a sequence representing a function on S.
Returns: Tv, a list."""
Tv = []
for x in S:
# Compute the value of the objective function for each
# a in Gamma(x), and store the result in vals (n*m matrix)
vals = []
for a in Gamma(x):
y = U(x - a) + rho * sum(v[a + z]*phi(z) for z in Z)
# the place v comes into play, v is array for each state
vals.append(y)
# Store the maximum reward for this x in the list Tv
Tv.append(max(vals))
return Tv
# create initial value
def v_init():
v = []
for i in S:
val = []
for j in Gamma(i):
# deterministic
y = U(i-j)
val.append(y)
v.append(max(val))
return v
# Create an instance of value function
v = v_init()
# parameters
max_iter = 10000
tol = 0.0001
num_iter = 0
diff = 1.0
N = len(S)
# value iteration
value = np.empty([max_iter,N])
while (diff>=tol and num_iter<max_iter ):
v = T(v)
value[num_iter] = v
diff = np.abs(value[-1] - value[-2]).max()
num_iter = num_iter + 1
As you can see, the while loop at the bottom is used to iterate over "value function" and find the right answer. However, the while fails to loop, and just return num_iter=1. As for I know, the while loop "repeats a sequence of statements until some condition becomes false", clearly, this condition will not be satisfied until the diff converge to near 0
The major part of code works just fine, as far as I use the following for loop
value = np.empty([num_iter,N])
for x in range(num_iter):
v = T(v)
value[x] = v
diff = np.abs(value[-1] - value[-2]).max()
print(diff)
You define value as np.empty(...). That means that it is composed completely of zeros. The difference, therefore, between the last element and the second-to-last element will be zero. 0 is not >= 0.0001, so that expression will be False. Therefore, your loop breaks.
I have an array of values, t, that is always in increasing order (but not always uniformly spaced). I have another single value, x. I need to find the index in t such that t[index] is closest to x. The function must return zero for x < t.min() and the max index (or -1) for x > t.max().
I've written two functions to do this. The first one, f1, is MUCH quicker in this simple timing test. But I like how the second one is just one line. This calculation will be done on a large array, potentially many times per second.
Can anyone come up with some other function with comparable timing to the first but with cleaner looking code? How about something quicker then the first (speed is most important)?
Thanks!
Code:
import numpy as np
import timeit
t = np.arange(10,100000) # Not always uniform, but in increasing order
x = np.random.uniform(10,100000) # Some value to find within t
def f1(t, x):
ind = np.searchsorted(t, x) # Get index to preserve order
ind = min(len(t)-1, ind) # In case x > max(t)
ind = max(1, ind) # In case x < min(t)
if x < (t[ind-1] + t[ind]) / 2.0: # Closer to the smaller number
ind = ind-1
return ind
def f2(t, x):
return np.abs(t-x).argmin()
print t, '\n', x, '\n'
print f1(t, x), '\n', f2(t, x), '\n'
print t[f1(t, x)], '\n', t[f2(t, x)], '\n'
runs = 1000
time = timeit.Timer('f1(t, x)', 'from __main__ import f1, t, x')
print round(time.timeit(runs), 6)
time = timeit.Timer('f2(t, x)', 'from __main__ import f2, t, x')
print round(time.timeit(runs), 6)
This seems much quicker (for me, Python 3.2-win32, numpy 1.6.0):
from bisect import bisect_left
def f3(t, x):
i = bisect_left(t, x)
if t[i] - x > 0.5:
i-=1
return i
Output:
[ 10 11 12 ..., 99997 99998 99999]
37854.22200356027
37844
37844
37844
37854
37854
37854
f1 0.332725
f2 1.387974
f3 0.085864
np.searchsorted is binary search (split the array in half each time). So you have to implement it in a way it return the last value smaller than x instead of returning zero.
Look at this algorithm (from here):
def binary_search(a, x):
lo=0
hi = len(a)
while lo < hi:
mid = (lo+hi)//2
midval = a[mid]
if midval < x:
lo = mid+1
elif midval > x:
hi = mid
else:
return mid
return lo-1 if lo > 0 else 0
just replaced the last line (was return -1). Also changed the arguments.
As the loops are written in Python, it may be slower than the first one... (Not benchmarked)
Use searchsorted:
t = np.arange(10,100000) # Not always uniform, but in increasing order
x = np.random.uniform(10,100000)
print t.searchsorted(x)
Edit:
Ah yes, I see that's what you do in f1. Maybe f3 below is easier to read than f1.
def f3(t, x):
ind = t.searchsorted(x)
if ind == len(t):
return ind - 1 # x > max(t)
elif ind == 0:
return 0
before = ind-1
if x-t[before] < t[ind]-x:
ind -= 1
return ind