Python/Theano: Is it possible to construct truly recursive theano functions? - python

For example, I can define a recursive Python lambda function for computing the Fibonacci sequence as follows:
fn = lambda z: fn(z-1)+fn(z-2) if z > 1 else z
However, if I try to turn this into a Theano function, Theano won't accept fn because fn invokes the Boolean operation ">". So this code crashes:
z = T.scalar('z')
fn = lambda z: fn(z-1)+fn(z-2) if z > 1 else z
fibby = theano.function([z], fn(z))
But if I replace the Boolean operator with theano.tensor.gt(z,1), the code goes into an infinite recursion, so theano.tensor.gt(z,1) isn't serving the role of ">":
z = T.scalar('z')
fn = lambda z: fn(z-1)+fn(z-2) if theano.tensor.gt(z,1) else z
lappy = theano.function([z], fn(z))
print(lappy(4))
Running this results in "maximum recursion depth exceeded". What's wrong? I get the same "maximum recursion depth exceeded" error if I replace the definition of fn with
fn = lambda z: theano.ifelse(theano.tensor.gt(z,1),fn(z-1)+fn(z-2),z)
PS I am NOT looking to do this using theano.scan... because I want to learn to do this calculation recursively without resorting to an explicit loop.
--Ken

In Theano you can make a recursion using outputs_info parameter of theano.scan(fn=myfunc(), outputs_info=...) and passing previous outputs of myfunc() as arguments in the next iteration of myfunc().
In case of Fibonacci sequence the code may look like this:
import numpy as np
import theano
import theano.tensor as T
# placeholder for the number of elements in the Fibonacci sequence
t_N = T.iscalar('N')
# first two elements for Fibonacci sequence
initial = np.array([1,1], dtype=np.int32)
t_initial = theano.shared(initial)
def fibonacci_iter(prev1_value, prev2_value):
return prev1_value + prev2_value
# Iterate N-2 times over fibonacci() function
# ('taps': [-2,-1] means take two previous values in the sequence of outputs):
outputs, updates = theano.scan(
fn=fibonacci_iter,
outputs_info = [{'initial': t_initial, 'taps': [-2,-1]}], # a list of dicts or a dict
n_steps=t_N-2)
# compiled function:
fibonacci = theano.function(
inputs=[t_N],
outputs=outputs)
n = 10
fibonacci_seq = fibonacci(n)
print(np.concatenate([initial, fibonacci_seq]))
Output:
[ 1 1 2 3 5 8 13 21 34 55]
Reference:
http://deeplearning.net/software/theano/library/scan.html#theano.scan

Related

How to pass a function as an argument

I would like to find an approximate value for the number pi = 3.14.. by using the Newton method. In order to use it also for some other purpose and thus other function than sin(x), the aim is to implement a generic function that will be passed over as an argument. I have an issue in passing a function as an argument into an other function. I also tried lambda in different variations. The code I am showing below produces the error message: IndexError: list index out of range. I will appreciate your help in solving this issue and eventually make any suggestion in the code which may not be correct. Thanks.
from sympy import *
import numpy as np
import math
x = Symbol('x')
# find the derivative of f
def deriv(f,x):
h = 1e-5
return (lambda x: (f(x+h)-f(x))/h)
def newton(x0,f,err):
A = [x0]
n = 1
while abs(A[n]-A[n-1])<=err:
if n == 1:
y = lambda x0: (math.f(x0))
b = x0-y(x0)/(deriv(y,x0))
A.append(b)
n += 1
else:
k = len(A)
xk = A[k]
y = lambda xk: (math.f(xk))
b = newton(A[k],y,err)-y(newton(A[k],y,err))/deriv(y,k)
A.append(b)
n += 1
return A, A[-1]
print(newton(3,math.sin(3),0.000001))
I don't know why you use sympy because I made it without Symbol
At the beginning you have to calculate second value and append it to list A and later you can calculate abs(A[n]-A[n-1]) (or the same without n: abs(A[-1] - A[-2])) because it needs two values from this list.
Other problem is that it has to check > instead of <=.
If you want to send function sin(x) then you have to use math.sin without () and arguments.
If you want to send function sin(3*x) then you would have to use lambda x: math.sin(3*x)
import math
def deriv(f, x, h=1e-5):
return (f(x+h) - f(x)) / h
def newton(x0, f, err):
A = [x0]
x = A[-1] # get last value
b = x - (f(x) / deriv(f, x)) # calculate new value
A.append(b) # add to list
while abs(A[-1] - A[-2]) > err: # it has to be `>` instead of `<=`
x = A[-1] # get last value
b = x - (f(x) / deriv(f, x)) # calculate new value
A.append(b) # add to list
return A, A[-1]
# sin(x)
print(newton(3, math.sin, 0.000001)) # it needs function's name without `()`
# sin(3*x)
print(newton(3, lambda x:math.sin(3*x), 0.000001))
# sin(3*x) # the same without `lambda`
def function(x):
return math.sin(3*x)
print(newton(3, function, 0.000001))
Result:
([3, 3.1425464414785056, 3.1415926532960112, 3.141592653589793], 3.141592653589793)
([3, 3.150770863559604, 3.1415903295877707, 3.1415926535897936, 3.141592653589793], 3.141592653589793)
EDIT:
You may write loop in newton in different way and it will need <=
def newton(x0, f, err):
A = [x0]
while True:
x = A[-1] # get last value
b = x - (f(x) / deriv(f, x)) # calculate new value
A.append(b) # add to list
if abs(A[-1] - A[-2]) <= err:
break
return A, A[-1]

Is it possible to update funcion variables every time it's been called?

I'm trying to solve an ODE system with solve_ivp and i want to change the local variables of the function every time it's been called by the solver.
In particular I wand to update the lagrange multipliers (lambdas_i) so that the next time step of solve_ivp, uses the values of the previous one.
(The ''reconstruct'' function is from a python module that uses a method to reconstruct a size distribution from given moments)
Is there a way to do this? I'll post the code below:
import time
import numpy as np
import scipy.integrate as integrate
from numpy import sqrt, sin, cos, pi
import math
import pylab as pp
from pymaxent import reconstruct
start = time.time()
'''Initialize variables'''
t=np.linspace(0, 60,31)
tspan=(0,60)
initial_m=[]
for i in range(4):
def distr(L,i=i):
return (L**i)*0.0399*np.exp(-((L-50)**2)/200)
m, err=integrate.quad(distr, 0, np.inf)
print('m(',i,')=',m)
initial_m.append(m)
''' Solving ode system using Maximum Entropy, G(L)=1+0.002*L'''
def moments(t,y):
m0 = y[0]
m1 = y[1]
m2 = y[2]
m3 = y[3]
Lmean=m1
σ=(m2-Lmean**2)**(1/2)
Lmin=Lmean-3*σ
Lmax=Lmean+4*σ
bnds=[Lmin,Lmax]
L=np.linspace(Lmin,Lmax)
sol, lambdas_i= reconstruct(mu=y ,bnds=bnds)
print(lambdas_i)
dm0dt=0
def moment1(L):
return(sol(L)+0.002*sol(L)*L)
dm1dt, err1=integrate.quad(moment1,Lmin,Lmax)
def moment2(L):
return(2*L*(sol(L)+0.002*sol(L)*L))
dm2dt, err2=integrate.quad(moment2,Lmin,Lmax)
def moment3(L):
return(3*L**2*(sol(L)+0.002*sol(L)*L))
dm3dt, err3=integrate.quad(moment3,Lmin,Lmax)
return(dm0dt,dm1dt,dm2dt,dm3dt)
'''Χρήση της BDF, step by step'''
r=integrate.solve_ivp(moments,tspan,initial_m,method='BDF',jac=None,t_eval=t,rtol=10**(-3))
end = time.time()
print('Total time =',{end-start})
Here is one way to achieve what you want. I won't be using your actual code, but using a simpler example problem, and you can use the same strategy to solve yours.
def seq():
l = [1, 0]
while True:
p = l[1]
n = l[0] + p
l = [p, n]
print(n)
input()
Above is an example function that would print the next (or first) term of the fibonacci sequence. (In which each term is the sum of the two previous terms.) In this case, the input is solely used to pause between each iteration (as it is infinite). Now to transform this into a generator, to allow more flexibility, you could rewrite the function as:
def seq():
l = [1, 0]
while True:
p = l[1]
n = l[0] + p
l = [p, n]
yield n
Now, if you wanted to get the same result, you could:
for item in seq():
print(item)
input()
However, this is mostly useless. The point of the generator comes in when you want to gather the next number of the sequence, but at any point of the code. Not necessarily inside the loop that repeats itself until you're done. You could achieve this:
gen = seq()
next(gen) # returns 1
next(gen) # returns 1
next(gen) # returns 2
next(gen) # returns 3
next(gen) # returns 5
And so on...
Another way to solve this problem would be to use a global variable instead of a local variable.
l = [1, 0]
def seq():
p = l[1]
n = l[0] + p
l[0] = p
l[1] = n
return n
The variable l is defined outside the function, not inside, so it will not be discarded when the function exits. (To run the code the same way as before:)
while True:
print(seq())
input()
Both of these should be implementable in your code.

How to fix TypeError: 'Add' object is not callable?

def f(x):
f='exp(x)-x-2'
y=eval(f)
print(y)
return y
def bissection(f,f_line,f_2lines,a,b,epsilon1,epsilon2):
x=a
result_a=f(a)
x=b
result_b=f(b)
if (f.evalf(a)*f.evalf(b)>=0):
print("Interval [a,b] does not contain a zero ")
exit()
zeta=min(epsilon1,epsilon2)/10
x=a
while(f_line(x)>0):
if(x<b or x>-b):
x=x+zeta
else:
stop
ak=a
bk=b
xk=(ak+bk)/2
k=0
if (f(xk)*f(ak)<0):
ak=ak
bk=xk
if (f(xk)*f(bk)<0):
ak=xk
bk=bk
k=k+1
from sympy import *
import math
x=Symbol('x')
f=exp(x)-x-2
f_line=f.diff(x)
f_2lines=f_line.diff(x)
print("Derivative of f:", f_line)
print("2nd Derivative of f:", f_2lines)
a=int(input('Beginning of interval: '))
b=int(input('End of interval: '))
epsilon1=input('1st tolerance: ')
epsilon2=input('2nd tolerance: ')
bissection(f,f_line,f_2lines,a,b,epsilon1,epsilon2)
This program is an attempt to implement the Bissection Method. I've tried writing two functions:
The first one, f, is supposed to receive the extremes of the interval that may or may not contain a root (a and b) and return the value of the function evaluated in this point.
The second one, bissection, should receive the function, the function's first and second derivatives, the extremes of the interval (a,b) and two tolerances (epsilon1,epsilon2).
What I want to do is pass each value a and b, one at a time, as arguments to the function f, that is supposed to return f(a) and f(b); that is, the values of the function in each of the points a and b.
Then, it should test two conditions:
1) If the function values in the extremes of the intervals have opposite signs. If they don't, the method won't converge for this interval, then the program should terminate.
if(f.evalf(a)*f.evalf(b)>=0)
exit()
2)
while(f_line(x)>0): #while the first derivative of the function evaluated in x is positive
if(x<b or x>-b): #This should test whether x belongs to the interval [a,b]
x=x+zeta #If it does, x should receive x plus zeta
else:
stop
At the end of this loop, my objective was to determine whether the first derivative was strictly positive (I didn't do the negative case yet).
The problem: I'm getting the error
Traceback (most recent call last):
File "bissec.py", line 96, in <module>
bissection(f,f_line,f_2lines,a,b,epsilon1,epsilon2)
File "bissec.py", line 41, in bissection
result_a=f(a)
TypeError: 'Add' object is not callable
How can I properly call the function so that it returns the value of the function (in this case, f(x)=exp(x)-x-2), for every x needed? That is, how can I evaluate f(a) and f(b)?
Ok, so I've figured it out where your program was failing and I've got 4 reasons why.
First of all, and the main topic of your question, if you want to evaluate a function f for a determined x value, let's say a, you need to use f.subs(x, a).evalf(), as it is described in SymPy documentation. You used in 2 different ways: f.evalf(2) and f_line(a); both were wrong and need to be substituted by the correct syntax.
Second, if you want to stop a while loop you should use the keyword break, not "stop", as written in your code.
Third, avoid using the same name for variables and functions. In your f function, you also used f as the name of a variable. In bissection function, you passed f as a parameter and tried to call the f function. That'll fail too. Instead, I've changed the f function to f_calc, and applied the correct syntax of my first point in it.
Fourth, your epsilon1 and epsilon2 inputs were missing a float() conversion. I've added that.
Now, I've also edited your code to use good practices and applied PEP8.
This code should fix this error that you're getting and a few others:
from sympy import *
def func_calc(func, x, val):
"""Evaluate a given function func, whose varible is x, with value val"""
return func.subs(x, val).evalf()
def bissection(x, f, f_line, f_2lines, a, b, epsilon1, epsilon2):
"""Applies the Bissection Method"""
result_a = func_calc(f, x, a)
result_b = func_calc(f, x, b)
if (result_a * result_b >= 0):
print("Interval [a,b] does not contain a zero")
exit()
zeta = min(epsilon1, epsilon2) / 10
x_val = a
while(func_calc(f_line, x, a) > 0):
if(-b < x_val or x_val < b):
x_val = x_val + zeta
else:
break # the keyword you're looking for is break, instead of "stop"
print(x_val)
ak = a
bk = b
xk = (ak + bk) / 2
k = 0
if (func_calc(f, x, xk) * func_calc(f, x, ak) < 0):
ak = ak
bk = xk
if (func_calc(f, x, xk) * func_calc(f, x, bk) < 0):
ak = xk
bk = bk
k = k + 1
def main():
x = Symbol('x')
f = exp(x) - x - 2
f_line = f.diff(x)
f_2lines = f_line.diff(x)
print("Derivative of f:", f_line)
print("2nd Derivative of f:", f_2lines)
a = int(input('Beginning of interval: '))
b = int(input('End of interval: '))
epsilon1 = float(input('1st tolerance: '))
epsilon2 = float(input('2nd tolerance: '))
bissection(x, f, f_line, f_2lines, a, b, epsilon1, epsilon2)
if __name__ == '__main__':
main()

Recursion limit in Python when indexing

I was fooling around with the recursion limit in Python, which may be dynamically altered using sys.setrecursionlimit(limit). The code below demonstrates that the integer limit exactly corresponds to the maximum depth allowed for recursive function calls.
For recursive indexing using [], the same recursion limit seems to apply, but apparently with a factor of 3, meaning that I can index three times deeper than I can call:
The above plot is generated by the code below.
import itertools, sys
import numpy as np
import matplotlib.pyplot as plt
limits = np.arange(10, 500, 100)
# Find max depth of recursive calls
maxdepth = []
for limit in limits:
sys.setrecursionlimit(limit)
try:
n = [0]
def fun(n=n):
n[0] += 1
fun()
fun()
except RecursionError:
maxdepth.append(n[0])
a, b = np.polyfit(limits, maxdepth, 1)
plt.plot(limits, maxdepth, '*')
plt.plot(limits, a*limits + b, '-', label='call')
plt.text(np.mean(limits), a*np.mean(limits) + b, f'slope = {a:.2f}')
# Find max depth of getitem
maxdepth = []
n = 0
l = []; l.append(l)
for limit in limits:
sys.setrecursionlimit(limit)
for n in itertools.count(n):
try:
eval('l' + '[-1]'*n)
except RecursionError:
break
maxdepth.append(n)
a, b = np.polyfit(limits, maxdepth, 1)
plt.plot(limits, maxdepth, '*')
plt.plot(limits, a*limits + b, '-', label='getitem')
plt.text(np.mean(limits), a*np.mean(limits) + b, f'slope = {a:.2f}')
plt.xlabel('Recursion limit')
plt.ylabel('Max depth')
plt.legend()
plt.savefig('test.png')
To test the recursive indexing I append a list l to itself and construct a long literal [-1][-1][-1]... which I then evaluate on l dynamically.
Question: Explain this factor of 3.
There’s no recursion in l[-1][-1]…—it compiles to “push l; replace top of stack with its last element; replace…”. Your RecursionError is from compiling the long string.
There is very literally a factor of 3 used to approximate the stack usage of the byte-compiler vs. the interpreter proper. (Python 2 has no such limit and just crashes on such expressions.)

Best way to find roots of a multidimensional, scalar function with SciPy

Suppose I have a function whose range is a scalar but whose domain is a vector. For example:
def func(x):
return x[0] + 1 + x[1]**2
What's a good way to find the a root of this function? scipy.optimize.fsolve and scipy.optimize.root expect func to return a vector (rather than a scalar), and scipy.optimize.newton only takes scalar arguments. I can redefine func as
def func(x):
return [x[0] + 1 + x[1]**2, 0]
Then root and fsolve can find a root, but the zeros in the Jacobian means it won't always do a good job. For example:
fsolve(func, array([0,2]))
=> array([-5, 2])
It'll only vary the first parameter but not the second, meaning that it often finds a zero that's far away.
EDIT: it looks like the following redefinition of func works better:
def func(x):
fx = x[0] + 1 + x[1]**2
return [fx, fx]
fsolve(func, array([0,5]))
=>array([-16.27342781, 3.90812331])
So it's now willing to change both parameters. The code is still kind of ugly though.
Have you tried the minimization of the absolute value of your function using fmin?
For example:
>>> import scipy.optimize as op
>>> import numpy as np
>>> def func(x):
>>> return x[0] + 1 + x[1]**2
>>> func1 = lambda x: np.abs(func(x))
>>> tmp = op.fmin(func1, [10000., 10000.])
>>> func(tmp)
0.0
>>> print tmp
[-8346.12025122 91.35162971]
Since -- for my problem -- I have a good initial guess and a non-crazy function, Newton's method works well. For a scalar, multidimensional function, Newton's method becomes:
Here's a rough code example:
def func(x): #the function to find a root of
return x[0] + 1 + x[1]**2
def dfunc(x): #the gradient of that function
return array([1, 2*x[1]])
def newtRoot(x0, func, dfunc):
x = array(x0)
for n in xrange(100): # do at most 100 iterations
f = func(x)
df = dfunc(x)
if abs(f) < 1e-6: # exit function if we're close enough
break
x = x - df*f/norm(df)**2 # update guess
return x
In use:
nsolve([0,2],func,dfunc)
=> array([-1.0052546 , 0.07248865])
func([-1.0052546 , 0.07248865])
=> 4.3788225025098715e-09
Not bad! Of course, this function is very rough, but you get the idea. It also won't work well for "tricky" functions or where you don't have a good starting guess. I think I'll use something like this but then fall back to fsolve or root if Newton's method doesn't converge.

Categories

Resources