Develop Reference

function of two variables reduced to 1 variable but issues with minimizing in Python

I have a function which is multivariate (code below), depending on x1 and x2 and I have a target output value in mind for this function, named target. I fix x1 and provide my target in the outer function: find_x2. The objective is to minimize the inner function error, which is just the squared distance between my output target and the resulting value est, with respect to x2.
But having run it:
find_x2(target = .12, x1 = 100) yields x2 = 0.99899, but it should be roughly .80.
It appears the function is minimizing with respect to est instead of x2, but I can't see why. Any help would be appreciated.
Code:
import numpy as np
from scipy.optimize import minimize_scalar, minimize
def find_x2(x1, target):
def error(x2, x1 = x1, target = target):
k = np.zeros(x1)
V = np.zeros(x1)
for i in range(x1):
try:
k[i-1] = (i/x1)**(-1/x2)
except:
k[x1-1]=1
for i in range(x1):
V[i] = k[i]/sum(k)
est = sum(V**2)
return (target - est)**2
return minimize_scalar(error, bounds = (.001,.999), method = 'bounded')

Try this version:
def find_x2(x1, target):
def error(x2, x1 = x1, target = target):
k = np.zeros(x1)
V = np.zeros(x1)
for i in range(x1):
try:
k[i-1] = (x1/i)**(1/x2)
except:
k[x1-1]=1
for i in range(x1):
V[i] = k[i]/sum(k)
est = sum(V**2)
return (target - est)**2
return minimize_scalar(error, bounds = (.001,.999), method = 'bounded')
find_x2(100, 0.12)
prints
fun: 9.668753433849857e-14
message: 'Solution found.'
nfev: 14
status: 0
success: True
x: 0.801701074855243

Locally bind functions within lambda's

Is there a way to locally bind functions within lambdas? I have a loop, and within each loop, I create an array of functions. I want to create another function which is the sum of all of these functions and store it in another array. This new array should hold the sum of all functions for each loop.
Then I want to create another function that is the sum over all the sum of functions.
However, the problem I have is that the original functions keep updating, so I am not getting my desired result. Can I locally bind the functions? Am I approaching this the wrong way?
import numpy as np
lmax = 4
lapprox = []
# Function to estimate
def curve_func(x):
return np.sin(x*np.pi)*x**2
# Initialise residual function
def residual_func(x):
return curve_func(x)
# For each l, create 2**l+1 nodes and determine new nodes.
for l in range(lmax):
nodes = np.linspace(0, 1, 2**l+1, endpoint = True)
if (l==0):
old_nodes = nodes
new_nodes = nodes
else:
old_nodes = np.linspace(0, 1, 2**(l-1)+1, endpoint = True)
new_nodes = [x for x in nodes if x not in old_nodes]
# Create basis function corresponding to each new node
if (l==0):
phi = [lambda x, i=i: max(1 - abs(2**l * x - i), 0) for i in range(len(new_nodes))]
else:
phi = [lambda x, i=i: max(1 - abs(2**l * x - (2*i+1)), 0) for i in range(len(new_nodes))]
# Calculate hierarchical surpluses
coeff = [residual_func(n) for n in new_nodes]
# Array of functions: coeff*phi
coeff_phi = [lambda x, func=func, alpha=alpha: coeff[alpha]*func(x) for alpha, func in enumerate(phi)]
# Array of length lmax, where each value is sum of functions in coeff_phi for fixed l
lapprox.append(lambda x: sum(f(x) for f in coeff_phi))
# Sum of all functions in lapprox
totapprox = lambda x: sum(f(x) for f in lapprox)
# Compute new residual function
residual_func = lambda x: curve_func(x) - totapprox(x)
Extra detail on what the code is doing: The code is designed to approximate a function, such as sin(pi*x)*x^2 using hierarchical linear splines. For each level l, there are some basis functions, given by the array phi. The function is approximated using a linear combination of some coefficients multiplied by these basis functions. The approximation is done sequentially, starting from a low-level, with few basis functions, until a high-level, with many basis functions. I need to keep track of the rolling approximation of the function to determine the values of the new coefficients.
Edit 2: I've defined the functions outside the loop. However, I am struggling in working out how to create a function to keep track of the residual_function. I have attached the 'dirty' solution that works as intended for lmax=3, but I would like to generalise it for any lmax. How can I do that?
def curve_func(x):
return np.sin(x*np.pi)*x**2
def residual_func_0(x):
return curve_func(x)
# Define nodes function
def make_nodes(l):
return np.linspace(0, 1, 2**l+1, endpoint = True)
# Define new_nodes function
def make_new_nodes(l):
if (l==0):
new_nodes = np.linspace(0, 1, 2**l+1, endpoint = True)
else:
old_nodes = np.linspace(0, 1, 2**(l-1)+1, endpoint = True)
new_nodes = [x for x in make_nodes(l) if x not in old_nodes]
return new_nodes
# Define basis functions
def make_basis(l, i):
if l == 0:
return lambda x: max(1 - abs(2**l * x - i), 0)
else:
return lambda x: max(1 - abs(2**l * x - (2*i+1)), 0)
# Define coeff*basis functions
def make_scaled_basis(alpha, fn):
return lambda x: alpha * fn(x)
new_nodes_0 = make_new_nodes(0)
new_nodes_1 = make_new_nodes(1)
new_nodes_2 = make_new_nodes(2)
new_nodes_3 = make_new_nodes(3)
phi_0 = [make_basis(0, i) for i in range(len(new_nodes_0))]
phi_1 = [make_basis(1, i) for i in range(len(new_nodes_1))]
phi_2 = [make_basis(2, i) for i in range(len(new_nodes_2))]
phi_3 = [make_basis(3, i) for i in range(len(new_nodes_3))]
coeff_0 = [curve_func(n) for n in new_nodes_0]
coeff_phi_0 = [make_scaled_basis(alpha, fn) for alpha, fn in zip(coeff_0, phi_0)]
residual_func_0 = lambda x: curve_func(x) - sum(f(x) for f in coeff_phi_0)
coeff_1 = [residual_func_0(n) for n in new_nodes_1]
coeff_phi_1 = [make_scaled_basis(alpha, fn) for alpha, fn in zip(coeff_1, phi_1)]
residual_func_1 = lambda x: residual_func_0(x) - sum(f(x) for f in coeff_phi_1)
coeff_2 = [residual_func_1(n) for n in new_nodes_2]
coeff_phi_2 = [make_scaled_basis(alpha, fn) for alpha, fn in zip(coeff_2, phi_2)]
residual_func_2 = lambda x: residual_func_1(x) - sum(f(x) for f in coeff_phi_2)
coeff_3 = [residual_func_2(n) for n in new_nodes_3]
coeff_phi_3 = [make_scaled_basis(alpha, fn) for alpha, fn in zip(coeff_3, phi_3)]
residual_func_3 = lambda x: residual_func_2(x) - sum(f(x) for f in coeff_phi_3)

A simple way to localize both l and i in the body of a function is to create a function that returns a function which closes over the local variables l and i.
For example:
def make_basis(l, i):
if l == 0:
return lambda x: max(1 - abs(2**l * x - i), 0)
else:
return lambda x: max(1 - abs(2**l * x - (2*i+1)), 0)
...
for l in range(lmax):
...
phi = [make_basis(l, i) for i in range(len(new_nodes))]
Loops do not create new scopes; only function bodies do.

How can I test linearity (superposition) & shift-invariance in Python?

I'm new to Python & writing a program that takes a function:
X = np.linspace(0,50)
F1 = np.sin(X)
Tests whether the function is linear (as in, exhibits superposition):
for i in range(1,10):
LT1 = i*(F1)
X = i*X
LT2 = F1
if np.all(LT1) == np.all(LT2):
Linear = 'This function is linear.'
elif np.all(LT1) != np.all(LT2):
Linear = 'This function is nonlinear.'
break
And tests whether the function is shift-invariant:
for j in range(1,10):
SI1 = (F1)-j
X = X-j
SI2 = F1
if np.all(SI1) == np.all(SI2):
SI = 'This function is shift-invariant.'
elif np.all(SI1) != np.all(SI2):
SI = 'This function is shift-variant.'
break
But my code calls all functions, LSI or not, linear & shift-variant. Is there a better way to run these tests? I've seen linear regression offered as a means of testing linearity, but upon trying it with a sine function it misclassified the function as nonlinear. I have also been unable to find any guidance whatsoever on testing shift-invariance.

Lets first define all the required functionality
import numpy as np
def check_linearity(T, X, a, b):
# T[a*x+b*x] = a*T[x] + b*T[x]
LHS = T(a*x + b*x)
RHS = a*T(x) + b*T(x)
tolerence = 1e-4
if np.sum(LHS-RHS) < tolerence:
print('Linear System')
return True
else:
print('Not a Linear System')
return False
def check_shift_invariance(T, X, tau):
# T[X] = T[X-tau]
LHS = T(X)
RHS = T(X-tau)
tolerence = 1e-4
if np.sum(LHS-RHS) < tolerence:
print('Shift Invariant System')
return True
else:
print('Not a Shift Invariant System')
return False
def check_LSI(T, X, a, b, tau):
flag1 = check_linearity(T, X, a, b)
flag2 = check_shift_invariance(T, X, tau)
if flag1== True and flag2==True:
print('LSI system')
else:
print('not a LSI system')
Next, we define signal
# Signal X in range [-1,1]
X = np.linspace(-1,1,10)
The, define System
# Transformation T
T = lambda x: np.sin(x)
Now lets see everything we defined in action
a = 1
b = 1
tau = 2*np.pi
# Check Linearity
check_linearity(T, X, a, b);
# Check Shift Invariance
check_shift_invariance(T, X, tau);
# Check LSI or not
check_LSI(T, X, a, b, tau);
You can easily define other systems like,
T = lambda x: x
T = lambda x: np.sin(x) + np.cos(x)
T = lambda x: x**2 + x + 2
and so on

#Ragnar provided a very nice mathematical solution, but I want to post a short one.
If your function is given by the values which are equispaced, then
print(all(abs(x) < 0.001 for x in np.diff(function, n=2)))
returns True if the function is linear and False otherwise.
The idea is that each time, the function is incremented by the same value (here I use that X is equispaced). Therefore, taking the difference of consecutive numbers should return the array with all identical entries. Taking the difference of the consecutive numbers again returns all zeros if the function is linear.

Root Finding Function for upper and lower intersections of the x-axis

I need to find the lower and upper intersections with the x-axis of a curve given by
y=f(x)=10⋅exp(sin(x))−(x^2)/2
In order to find the arc length of the curve, in Python
I have already tried two methods, The secant method which I cannot get to work at all. And the Newton Method which finds one intersection.
from math import exp
from math import sin
from math import cos
def func( x ):
return 10*exp(sin(x))-(x**2)/2
def derivFunc( x ):
return 10*exp(sin(x))*cos(x)-x
def newtonRaphson( x ):
h = func(x) / derivFunc(x)
while abs(h) >= 0.0001:
h = func(x)/derivFunc(x)
x = x - h
print("The value of the root is : ",
"%.4f"% x)
x0 = -20
newtonRaphson(x0)
which gives
The value of the root is : -5.7546
Then the second method
import math
from math import exp
from math import sin
def f(x):
f = 10*exp(sin(x))-(x**2)/2
return f;
def secant(x1, x2, E):
n = 0; xm = 0; x0 = 0; c = 0;
if (f(x1) * f(x2) < 0):
while True:
x0 = ((x1 * f(x2) - x2 * f(x1)) /(f(x2) - f(x1)));
c = f(x1) * f(x0);
x1 = x2;
x2 = x0;
n += 1;
if (c == 0):
xm = ((x1 * f(x2) - x2 * f(x1)) /(f(x2) - f(x1)));
if(abs(xm - x0) < E):
print("Root of the given equation =",round(x0, 6));
print("No. of iterations = ", n);
print("Can not find a root in ","the given inteval");
x1 = 0; x2 = 1;
E = 0.0001;
secant(x1, x2, E);
Only results in
NameError: name 'x2' is not defined
Yet whenever I've tried defining the characters it won't run
I would like to be able to get the upper and lower intersections with the x-axis, So I can find the arc length. And is there a way to get it to plot the graph as well

About the Newton-Raphson method:
Normal behaviour
It works mostly as intended. The method may converge to a single root only, which depends on the starting point. To get another root, you need another starting point.
Your function yields:
>>> newtonRaphson(-20)
-5.7545790362989
>>> newtonRaphson(5)
3.594007784799419
Which seems to be correct.
Bugs
The Newton-Raphson method isn't guaranteed to converge, it may enter an ifinite loop, in which case your program would hang indefinitely, or the derivative at a point may be zero, in which case you can't compute h. You need to handle these cases.
Style
There is a lot of things that can be inproved:
The bug must be fixed
You Newton-Raphson method currently only works for one function only. You should pass the function and derivative as arguments, so you can apply the method to any function you want.
The desired precision and max iterations can also be passed as arguments
It's bad practice to print within the function. You should return the value instead, so you can decide to do whatever with the result.
you should follow PEP8's style guidelines
include a docstring if you plan to reuse it (wich is very possible, it is a very useful tool!)
My take on the method:
def newton_raphson(f, df, x, epsilon = 0.0001, maxiter = 1000):
""" Estimates the root of a function.
Gives an estimate to the required precision of a root of the given function
using the Newton-Raphson method.
Raises an Exception if the Newton-Raphson method doesn't converge in the
specified number of iterations.
Raises a ZeroDivisionError if the derivative is zero at a calculated point
:param f: The function
:param df: The function's derivative
:param x: the starting point for the method
:param epsilon: The desired precision
:param maxiter: The maximum number of iterations
:return: The root extimate
:rtype: float
"""
for _ in range(maxiter):
h = f(x)/df(x)
if abs(h) < epsilon:
return x
x = x - h
raise Exception("Newton Raphson method didn't "
+ "converge in {} iterations".format(maxiter))
usage:
>>> print(newton_raphson(func, derivFunc, 20))
-5.7545790362989
>>> print(newton_raphson(func, derivFunc, 5, 0.1, 100))
3.5837828560043477
>>> print(newton_raphson(func, derivFunc, 5, 0.001, 100))
3.594007784799419
>>> print(newton_raphson(func, derivFunc, 5, 1e-9, 4))
Traceback (most recent call last):
(...)
Exception: Newton Raphson method didn't converge in 4 iterations
About the secant method:
I'm not as familiar with that one, so I'll just mention the error you have is due to bad identation. Here it is fixed:
def secant(x1, x2, E):
n = 0; xm = 0; x0 = 0; c = 0;
if (f(x1) * f(x2) < 0):
while True:
x0 = ((x1 * f(x2) - x2 * f(x1)) /(f(x2) - f(x1)));
c = f(x1) * f(x0);
x1 = x2;
x2 = x0;
n += 1;
if (c == 0):
xm = ((x1 * f(x2) - x2 * f(x1)) /(f(x2) - f(x1)));
if(abs(xm - x0) < E):
print("Root of the given equation =",round(x0, 6));
print("No. of iterations = ", n);
print("Can not find a root in ","the given inteval");
If you plan to propely implement this method, the remarks about the Newton-Raphson method still hold.

What is faster, scipy fsolve vs root?

I am trying to find the root of a function. I have used fsolve in the past but as my data sets get larger, it seems to get more inconsistent (--> n = 187). Now I am looking for alternatives and have found scipy.root. I don't understand what the difference is between the two, and which one is better in my scenario.
I am trying to solve the following 3N coupled equations and trying to find vector x y and z:
My code is the following, where inrec, outrec and rec are predetermined lists of values:
import scipy as sp
import numpy as np
from scipy.optimize import fsolve
import math
def f(w, n, onrec, inrec, rec):
F = [0]*3*n
for i in range(n):
F[i] = -onrec[i] #k_i>
F[n+i] = -inrec[i] #k_i<
F[(2*n)+i] = -rec[i] #k_i <>
for j in range(n):
if i == j:
None
else: #below the three functions are stated. w[i] = x_i, w[n+i] = y_i, w[2*n + i] = z_i
F[i] += (w[i]*w[n+j])/(1+w[i]*w[n+j]+w[j]*w[n+i]+w[2*n+i]*w[2*n+j])
F[n+i] += (w[j]*w[n+i])/(1+w[i]*w[n+j]+w[j]*w[n+i]+w[2*n+i]*w[2*n+j])
F[2*n+i] += (w[(2*n)+i]*w[(2*n)+j])/(1+w[i]*w[n+j]+w[j]*w[n+i]+w[2*n+i]*w[2*n+j])
return(F)
u = [1]*3*n
s = fsolve(f, u, args=(n, onrec, inrec, rec))

As #Bob suggests, step 1 must be to vectorise your inner function. After that, to your main question: it's not the right thing to ask, because
fsolve is just a wrapper around the hybr algorithm, which is already provided as one option in root; and
worry about correctness before performance.
It's almost certain that the optimiser is giving up on your problem and the results are invalid. The only circumstance under which I was able to convince it to converge was with n=4 and the Levenberg-Marquardt algorithm. If (four years later) you still need to solve this, I recommend bringing the problem to a different community like Mathematics StackExchange. In the meantime, though, here is a vectorised example with one converging solution:
import numpy as np
from numpy.random import default_rng
from scipy.optimize import root
def lhs(xyz: np.ndarray) -> np.ndarray:
x, y, z = xyz[..., np.newaxis]
n = len(x)
coeff = (1 - np.eye(n)) / (1 + x*y.T + x.T*y + z*z.T)
numerator = np.stack((
x * y.T,
x.T * y,
z.T * z,
))
result = (numerator * coeff).sum(axis=2)
return result
def to_root(w: np.ndarray, recs: np.ndarray) -> np.ndarray:
xyz = w.reshape((3, -1))
rhs = lhs(xyz) - recs
return rhs.ravel()
def test_data(n: int = 4) -> tuple[np.ndarray, np.ndarray]:
rand = default_rng(seed=0)
secret_solution = rand.uniform(-1, 1, (3, n))
recs = lhs(secret_solution)
return secret_solution, recs
def method_search() -> None:
secret_solution, recs = test_data()
for method in ('hybr', 'lm', 'broyden1', 'broyden2', 'anderson',
'linearmixing', 'diagbroyden', 'excitingmixing',
'krylov', 'df-sane'):
try:
result = root(
to_root, x0=np.ones_like(recs),
args=(recs,), method=method,
options={
'maxiter': 5_000,
'maxfev': 5_000,
},
)
except Exception:
continue
print(method, result.message,
f'nfev={getattr(result, "nfev", None)} nit={getattr(result, "nit", None)}')
print('Estimated RHS:')
print(lhs(result.x.reshape((3, -1))))
print('Estimated error:')
print(to_root(result.x, recs))
print()
def successful_example() -> None:
n = 4
print('n=', n)
secret_solution, recs = test_data(n=n)
result = root(
to_root, x0=np.ones_like(recs),
args=(recs,), method='lm',
)
print(result.message)
print('function evaluations:', result.nfev)
error = to_root(result.x, recs)
print('Error:', error.dot(error))
print()
if __name__ == '__main__':
successful_example()
n= 4
The relative error between two consecutive iterates is at most 0.000000
function evaluations: 1221
Error: 8.721381160163159e-30