Develop Reference

How to use Gradient Descent to solve this multiple terms trigonometry function?

Question is like this:
f(x) = A sin(2π * L * x) + B cos(2π * M * x) + C sin(2π * N * x)
and L,M,N are constants integer, 0 <= L,M,N <= 100
and A,B,C can be any possible integers.
Here is the given data:
x = [0,0.01,0.02,0.03,0.04,0.05,0.06,0.07,0.08,0.09,0.1,0.11,0.12,0.13,0.14,0.15,0.16,0.17,0.18,0.19,0.2,0.21,0.22,0.23,0.24,0.25,0.26,0.27,0.28,0.29,0.3,0.31,0.32,0.33,0.34,0.35,0.36,0.37,0.38,0.39,0.4,0.41,0.42,0.43,0.44,0.45,0.46,0.47,0.48,0.49,0.5,0.51,0.52,0.53,0.54,0.55,0.56,0.57,0.58,0.59,0.6,0.61,0.62,0.63,0.64,0.65,0.66,0.67,0.68,0.69,0.7,0.71,0.72,0.73,0.74,0.75,0.76,0.77,0.78,0.79,0.8,0.81,0.82,0.83,0.84,0.85,0.86,0.87,0.88,0.89,0.9,0.91,0.92,0.93,0.94,0.95,0.96,0.97,0.98,0.99]
y = [4,1.240062433,-0.7829654986,-1.332487982,-0.3337640721,1.618033989,3.512512389,4.341307895,3.515268061,1.118929599,-2.097886967,-4.990538967,-6.450324073,-5.831575611,-3.211486891,0.6180339887,4.425660706,6.980842552,7.493970785,5.891593744,2.824429495,-0.5926374511,-3.207870455,-4.263694544,-3.667432785,-2,-0.2617162175,0.5445886005,-0.169441247,-2.323237059,-5.175570505,-7.59471091,-8.488730333,-7.23200463,-3.924327772,0.6180339887,5.138501587,8.38127157,9.532377045,8.495765687,5.902113033,2.849529206,0.4768388529,-0.46697525,0.106795821,1.618033989,3.071952496,3.475795162,2.255463709,-0.4905371745,-4,-7.117914956,-8.727599664,-8.178077181,-5.544088451,-1.618033989,2.365340134,5.169257268,5.995297102,4.758922924,2.097886967,-0.8873135564,-3.06024109,-3.678989552,-2.666365632,-0.6180339887,1.452191817,2.529722611,2.016594378,-0.01374122059,-2.824429495,-5.285215072,-6.302694708,-5.246870619,-2.210419738,2,6.13956874,8.965976562,9.68000641,8.201089581,5.175570505,1.716858387,-1.02183483,-2.278560533,-1.953524751,-0.6180339887,0.7393509358,1.129293593,-0.02181188158,-2.617913164,-5.902113033,-8.727381729,-9.987404016,-9.043589913,-5.984648344,-1.618033989,2.805900027,6.034770001,7.255101454,6.368389697]
enter image description here
How to use Gradient Descent to solve this multiple terms trigonometry function?

Gradient descent is not well suited for optimisation over integers. You can try a navie relaxation where you solve in floats, and hope the rounded solution is still ok.
from autograd import grad, numpy as jnp
import numpy as np
def cast(params):
[A, B, C, L, M, N] = params
L = jnp.minimum(jnp.abs(L), 100)
M = jnp.minimum(jnp.abs(M), 100)
N = jnp.minimum(jnp.abs(N), 100)
return A, B, C, L, M, N
def pred(params, x):
[A, B, C, L, M, N] = cast(params)
return A *jnp.sin(2 * jnp.pi * L * x) + B*jnp.cos(2*jnp.pi * M * x) + C * jnp.sin(2 * jnp.pi * N * x)
x = [0,0.01,0.02,0.03,0.04,0.05,0.06,0.07,0.08,0.09,0.1,0.11,0.12,0.13,0.14,0.15,0.16,0.17,0.18,0.19,0.2,0.21,0.22,0.23,0.24,0.25,0.26,0.27,0.28,0.29,0.3,0.31,0.32,0.33,0.34,0.35,0.36,0.37,0.38,0.39,0.4,0.41,0.42,0.43,0.44,0.45,0.46,0.47,0.48,0.49,0.5,0.51,0.52,0.53,0.54,0.55,0.56,0.57,0.58,0.59,0.6,0.61,0.62,0.63,0.64,0.65,0.66,0.67,0.68,0.69,0.7,0.71,0.72,0.73,0.74,0.75,0.76,0.77,0.78,0.79,0.8,0.81,0.82,0.83,0.84,0.85,0.86,0.87,0.88,0.89,0.9,0.91,0.92,0.93,0.94,0.95,0.96,0.97,0.98,0.99]
y = [4,1.240062433,-0.7829654986,-1.332487982,-0.3337640721,1.618033989,3.512512389,4.341307895,3.515268061,1.118929599,-2.097886967,-4.990538967,-6.450324073,-5.831575611,-3.211486891,0.6180339887,4.425660706,6.980842552,7.493970785,5.891593744,2.824429495,-0.5926374511,-3.207870455,-4.263694544,-3.667432785,-2,-0.2617162175,0.5445886005,-0.169441247,-2.323237059,-5.175570505,-7.59471091,-8.488730333,-7.23200463,-3.924327772,0.6180339887,5.138501587,8.38127157,9.532377045,8.495765687,5.902113033,2.849529206,0.4768388529,-0.46697525,0.106795821,1.618033989,3.071952496,3.475795162,2.255463709,-0.4905371745,-4,-7.117914956,-8.727599664,-8.178077181,-5.544088451,-1.618033989,2.365340134,5.169257268,5.995297102,4.758922924,2.097886967,-0.8873135564,-3.06024109,-3.678989552,-2.666365632,-0.6180339887,1.452191817,2.529722611,2.016594378,-0.01374122059,-2.824429495,-5.285215072,-6.302694708,-5.246870619,-2.210419738,2,6.13956874,8.965976562,9.68000641,8.201089581,5.175570505,1.716858387,-1.02183483,-2.278560533,-1.953524751,-0.6180339887,0.7393509358,1.129293593,-0.02181188158,-2.617913164,-5.902113033,-8.727381729,-9.987404016,-9.043589913,-5.984648344,-1.618033989,2.805900027,6.034770001,7.255101454,6.368389697]
def loss(params):
p = pred(params, np.array(x))
return jnp.mean((np.array(y)-p)**2)
params = np.array([np.random.random()*100 for _ in range(6)])
for _ in range(10000):
g = grad(loss)
params = params - 0.001*g(params)
print("Relaxed solution", cast(params), "loss=", loss(params))
constrained_params = np.round(cast(params))
print("Integer solution", constrained_params, "loss=", loss(constrained_params))
print()
Since the problem will have a lot of local minima, you might need to run it multiple times.

It's quite hard to use gradient descent to find a solution to this problem, because it tends to get stuck when changing the L, M, or N parameters. The gradients for those can push it away from the right solution, unless it is very close to an optimal solution already.
There are ways to get around this, such as basinhopping or random search, but because of the function you're trying to learn, you have a better alternative.
Since you're trying to learn a sinusoid function, you can use an FFT to find the frequencies of the sine waves. Once you have those frequencies, you can find the amplitudes and phases used to generate the same sine wave.
Pardon the messiness of this code, this is my first time using an FFT.
import scipy.fft
import numpy as np
import math
import matplotlib.pyplot as plt
def get_top_frequencies(x, y, num_freqs):
x = np.array(x)
y = np.array(y)
# Find timestep (assume constant timestep)
dt = abs(x[0] - x[-1]) / (len(x) - 1)
# Take discrete FFT of y
spectral = scipy.fft.fft(y)
freq = scipy.fft.fftfreq(y.shape[0], d=dt)
# Cut off top half of frequencies. Assumes input signal is real, and not complex.
spectral = spectral[:int(spectral.shape[0] / 2)]
# Double amplitudes to correct for cutting off top half.
spectral *= 2
# Adjust amplitude by sampling timestep
spectral *= dt
# Get ampitudes for all frequencies. This is taking the magnitude of the complex number
spectral_amplitude = np.abs(spectral)
# Pick frequencies with highest amplitudes
highest_idx = np.argsort(spectral_amplitude)[::-1][:num_freqs]
# Find amplitude, frequency, and phase components of each term
highest_amplitude = spectral_amplitude[highest_idx]
highest_freq = freq[highest_idx]
highest_phase = np.angle(spectral[highest_idx]) / math.pi
# Convert it into a Python function
function = ["def func(x):", "return ("]
for i, components in enumerate(zip(highest_amplitude, highest_freq, highest_phase)):
amplitude, freq, phase = components
plus_sign = " +" if i != (num_freqs - 1) else ""
term = f"{amplitude:.2f} * math.cos(2 * math.pi * {freq:.2f} * x + math.pi * {phase:.2f}){plus_sign}"
function.append(" " + term)
function.append(")")
return "\n ".join(function)
x = [0,0.01,0.02,0.03,0.04,0.05,0.06,0.07,0.08,0.09,0.1,0.11,0.12,0.13,0.14,0.15,0.16,0.17,0.18,0.19,0.2,0.21,0.22,0.23,0.24,0.25,0.26,0.27,0.28,0.29,0.3,0.31,0.32,0.33,0.34,0.35,0.36,0.37,0.38,0.39,0.4,0.41,0.42,0.43,0.44,0.45,0.46,0.47,0.48,0.49,0.5,0.51,0.52,0.53,0.54,0.55,0.56,0.57,0.58,0.59,0.6,0.61,0.62,0.63,0.64,0.65,0.66,0.67,0.68,0.69,0.7,0.71,0.72,0.73,0.74,0.75,0.76,0.77,0.78,0.79,0.8,0.81,0.82,0.83,0.84,0.85,0.86,0.87,0.88,0.89,0.9,0.91,0.92,0.93,0.94,0.95,0.96,0.97,0.98,0.99]
y = [4,1.240062433,-0.7829654986,-1.332487982,-0.3337640721,1.618033989,3.512512389,4.341307895,3.515268061,1.118929599,-2.097886967,-4.990538967,-6.450324073,-5.831575611,-3.211486891,0.6180339887,4.425660706,6.980842552,7.493970785,5.891593744,2.824429495,-0.5926374511,-3.207870455,-4.263694544,-3.667432785,-2,-0.2617162175,0.5445886005,-0.169441247,-2.323237059,-5.175570505,-7.59471091,-8.488730333,-7.23200463,-3.924327772,0.6180339887,5.138501587,8.38127157,9.532377045,8.495765687,5.902113033,2.849529206,0.4768388529,-0.46697525,0.106795821,1.618033989,3.071952496,3.475795162,2.255463709,-0.4905371745,-4,-7.117914956,-8.727599664,-8.178077181,-5.544088451,-1.618033989,2.365340134,5.169257268,5.995297102,4.758922924,2.097886967,-0.8873135564,-3.06024109,-3.678989552,-2.666365632,-0.6180339887,1.452191817,2.529722611,2.016594378,-0.01374122059,-2.824429495,-5.285215072,-6.302694708,-5.246870619,-2.210419738,2,6.13956874,8.965976562,9.68000641,8.201089581,5.175570505,1.716858387,-1.02183483,-2.278560533,-1.953524751,-0.6180339887,0.7393509358,1.129293593,-0.02181188158,-2.617913164,-5.902113033,-8.727381729,-9.987404016,-9.043589913,-5.984648344,-1.618033989,2.805900027,6.034770001,7.255101454,6.368389697]
print(get_top_frequencies(x, y, 3))
That produces this function:
def func(x):
return (
5.00 * math.cos(2 * math.pi * 10.00 * x + math.pi * 0.50) +
4.00 * math.cos(2 * math.pi * 5.00 * x + math.pi * -0.00) +
2.00 * math.cos(2 * math.pi * 3.00 * x + math.pi * -0.50)
)
Which is not quite the format you specified - you asked for two sins and one cos, and for no phase parameter. However, using the trigonometric identity cos(x) = sin(pi/2 - x), you can convert this into an equivalent expression that matches what you want:
def func(x):
return (
5.00 * math.sin(2 * math.pi * -10.00 * x) +
4.00 * math.cos(2 * math.pi * 5.00 * x) +
2.00 * math.sin(2 * math.pi * 3.00 * x)
)
And there's the original function!

Error: TypeError: can't multiply sequence by non-int of type 'numpy.float64' <Figure size 432x288 with 0 Axes> (don't know what to do)

I'm trying to plot an equation that contains and definite integral. It is a photoionisation cross section associated with the intersubband transitions in a two-dimensional quantum ring. I made the angular part analytically, and I'm trying to calculate numerically the radial part.
Here's my attempt to implement this in a Python code:
from scipy.integrate import quad
import numpy as np
from scipy.special import gamma
from scipy.constants import alpha
import matplotlib.pyplot as plt
#Constants
epsilon = 13.1 #dielectric constant of the material
gamma_C = 0.5 # donor impurity linewidth
nr = 3.2 #refractive index of semiconductor
flux = 0 # Phi in eqn 8 magnetic flux
R = 5 #radius of the qunatum ring in nm
r = np.linspace(0, 6 * R)
rho = r / R
m_0 = 0.0067*0.511 # electron effective mass
h = 4.13e-15 # Planck constant in eV
hbar = 6.58e-16 # reduced Planck constant in eV
#Photon energy
hnu = np.linspace(0, 100) #in eV
#Function that calculates the integrand
def func(rho):
betai = np.sqrt( gama**4/4)
betaf = np.sqrt(1+gama**4/2)
return ((gama * rho)**(betai + betaf) *
np.exp(-1/2*(gama * rho)**2)
* (gama * rho)**2/2 )
def cross_section(hnu, gama):
#function that calculates the photoionisation cross section
betai = np.sqrt( gama**4/4)
betaf = np.sqrt(1+gama**4/2)
Ei = gama**2*(1+betai)-gama**4/2
Ef = gama**2*(3+betaf)-gama**4/2
return (nr/epsilon * 4*np.pi/3 * alpha * hnu *
(abs(R * np.sqrt(1/2**betai*gamma(betai + 1))*
np.sqrt(1/2**betaf*gamma(betaf + 1)) *
quad(func, 0, np.infty))**2 *
hbar * gamma_C/(Ef - Ei - hnu)**2 + ( hbar * gamma_C)**2))
#Plot
plt.figure();plt.clf()
for gama in [1.0, 1.5, 2.0]:
plt.plot(hnu, cross_section(hnu, gama))
But I keep receiving this error
TypeError: can't multiply sequence by non-int of type 'numpy.float64'
Anyone knows the cause and how can avoid this?

Take another look at the docstring for scipy.integrate.quad. In particular, look at the 'Returns' section. You'll see that it returns multiple values. More precisely, it returns a tuple of values. The actual number of values depends on the parameter full_output, but it always includes at least two values, the numerically computed integral and the error estimate.
In this code
return (nr/epsilon * 4*np.pi/3 * alpha * hnu *
(abs(R * np.sqrt(1/2**betai*gamma(betai + 1))*
np.sqrt(1/2**betaf*gamma(betaf + 1)) *
quad(func, 0, np.infty))**2 *
hbar * gamma_C/(Ef - Ei - hnu)**2 + ( hbar * gamma_C)**2))
you use the return value of quad, but that is a tuple, so it won't work correctly in that expression. To fix it, just pull out the first value of the tuple returned by quad. That is, replace quad(func, 0, np.infty) with quad(func, 0, np.infty)[0]:
return (nr/epsilon * 4*np.pi/3 * alpha * hnu *
(abs(R * np.sqrt(1/2**betai*gamma(betai + 1))*
np.sqrt(1/2**betaf*gamma(betaf + 1)) *
quad(func, 0, np.infty)[0])**2 *
hbar * gamma_C/(Ef - Ei - hnu)**2 + ( hbar * gamma_C)**2))

Derivative of function of vector-functions based on derivative of vectors in sympy

I want to calculate the derivative of a function of vector-functions based on derivative of vectors in Python using sympy. For example, if the function is:
f(g) = g(t) / (gT(t) * g(t)) ^ 0.5
then I want to have:
df/dt = [dg(t) / dt * (gT(t) * g(t)) ^ 0.5 - 0.5 * g(t) * (dgT(t) / dt * g(t) + gT(t) * dg(t) / dt) * (gT(t) * g(t)) ^ (-0.5)] / [gT(t) * g(t)]
In which gT(t) is transpose of the vector function g(t). How can I do that?

The derivative can be calculated with
>>> from sympy.abc import t
>>> g = Function('g', commutative=False)
>>> gT = Function('gT', commutative=False)
>>> f = g(t) / sqrt(gT(t) * g(t))
>>> f.diff(t).simplify()
(-g(t)*(gT(t)*g(t))**(-1/2)*(gT(t)*Derivative(g(t), t) + Derivative(gT(t), t)*g(t))*(gT(t)*g(t))**(-1) + 2*Derivative(g(t), t)*(gT(t)*g(t))**(-1/2))/2

using fsolve to find the solution

import numpy as np
from scipy.optimize import fsolve
musun = 132712000000
T = 365.25 * 86400 * 2 / 3
e = 581.2392124070273
def f(x):
return ((T * musun ** 2 / (2 * np.pi)) ** (1 / 3) * np.sqrt(1 - x ** 2)
- np.sqrt(.5 * musun ** 2 / e * (1 - x ** 2)))
x = fsolve(f, 0.01)
f(x)
print x
What is wrong with this code? It seems to not work.

Because sqrt returns NaN for negative argument, your function f(x) is not calculable for all real x. I changed your function to use numpy.emath.sqrt() which can output complex values when the argument < 0, and returns the absolute value of the expression.
import numpy as np
from scipy.optimize import fsolve
sqrt = np.emath.sqrt
musun = 132712000000
T = 365.25 * 86400 * 2 / 3
e = 581.2392124070273
def f(x):
return np.abs((T * musun ** 2 / (2 * np.pi)) ** (1 / 3) * sqrt(1 - x ** 2)
- sqrt(.5 * musun ** 2 / e * (1 - x ** 2)))
x = fsolve(f, 0.01)
x, f(x)
Then you can get the right result:
(array([ 1.]), array([ 121341.22302275]))
the solution is very close to the true root, but f(x) is still very large because f(x) has a very large factor: musun.

fsolve() returns the roots of f(x) = 0 (see here).
When I plotted the values of f(x) for x in the range -1 to 1, I found that there are roots at x = -1 and x = 1. However, if x > 1 or x < -1, both of the sqrt() functions will be passed a negative argument, which causes the error invalid value encountered in sqrt.
It doesn't surprise me that fsolve() fails to find roots that are at the very ends of the valid range for the function.
I find that it is always a good idea to plot the graph of a function before trying to find its roots, as that can indicate how likely (or in this case, unlikely) it is that the roots will be found by any root-finding algorithm.

python numerically solving an equation with no sign change between the upper and lower bound

I am trying to solve an equation but I am unable to use brentq since there is no sign change. How could I find the value of r?
>>> import numpy as np
>>>
>>> def f(r):
return 0.1 + 1 / (2 * r ** 2) - 2 / (3 * np.sqrt(r ** 3))

I think there is sign change, but you can solve it by fsolve:
from scipy.optimize import fsolve
import numpy as np
def f(r):
return 0.1 + 1 / (2 * r ** 2) - 2 / (3 * np.sqrt(r ** 3))
r = fsolve(f, 1)
f(r)
result:
r = 2.22213541
Edit
There are two solution:
import numpy as np
def f(r):
return 0.1 + 1 / (2 * r ** 2) - 2 / (3 * np.sqrt(r ** 3))
r = np.linspace(0.6, 3, 1000)
plot(r, f(r))
output: