Develop Reference

How to use Gradient Descent to solve this multiple terms trigonometry function?

Question is like this:
f(x) = A sin(2π * L * x) + B cos(2π * M * x) + C sin(2π * N * x)
and L,M,N are constants integer, 0 <= L,M,N <= 100
and A,B,C can be any possible integers.
Here is the given data:
x = [0,0.01,0.02,0.03,0.04,0.05,0.06,0.07,0.08,0.09,0.1,0.11,0.12,0.13,0.14,0.15,0.16,0.17,0.18,0.19,0.2,0.21,0.22,0.23,0.24,0.25,0.26,0.27,0.28,0.29,0.3,0.31,0.32,0.33,0.34,0.35,0.36,0.37,0.38,0.39,0.4,0.41,0.42,0.43,0.44,0.45,0.46,0.47,0.48,0.49,0.5,0.51,0.52,0.53,0.54,0.55,0.56,0.57,0.58,0.59,0.6,0.61,0.62,0.63,0.64,0.65,0.66,0.67,0.68,0.69,0.7,0.71,0.72,0.73,0.74,0.75,0.76,0.77,0.78,0.79,0.8,0.81,0.82,0.83,0.84,0.85,0.86,0.87,0.88,0.89,0.9,0.91,0.92,0.93,0.94,0.95,0.96,0.97,0.98,0.99]
y = [4,1.240062433,-0.7829654986,-1.332487982,-0.3337640721,1.618033989,3.512512389,4.341307895,3.515268061,1.118929599,-2.097886967,-4.990538967,-6.450324073,-5.831575611,-3.211486891,0.6180339887,4.425660706,6.980842552,7.493970785,5.891593744,2.824429495,-0.5926374511,-3.207870455,-4.263694544,-3.667432785,-2,-0.2617162175,0.5445886005,-0.169441247,-2.323237059,-5.175570505,-7.59471091,-8.488730333,-7.23200463,-3.924327772,0.6180339887,5.138501587,8.38127157,9.532377045,8.495765687,5.902113033,2.849529206,0.4768388529,-0.46697525,0.106795821,1.618033989,3.071952496,3.475795162,2.255463709,-0.4905371745,-4,-7.117914956,-8.727599664,-8.178077181,-5.544088451,-1.618033989,2.365340134,5.169257268,5.995297102,4.758922924,2.097886967,-0.8873135564,-3.06024109,-3.678989552,-2.666365632,-0.6180339887,1.452191817,2.529722611,2.016594378,-0.01374122059,-2.824429495,-5.285215072,-6.302694708,-5.246870619,-2.210419738,2,6.13956874,8.965976562,9.68000641,8.201089581,5.175570505,1.716858387,-1.02183483,-2.278560533,-1.953524751,-0.6180339887,0.7393509358,1.129293593,-0.02181188158,-2.617913164,-5.902113033,-8.727381729,-9.987404016,-9.043589913,-5.984648344,-1.618033989,2.805900027,6.034770001,7.255101454,6.368389697]
enter image description here
How to use Gradient Descent to solve this multiple terms trigonometry function?

Gradient descent is not well suited for optimisation over integers. You can try a navie relaxation where you solve in floats, and hope the rounded solution is still ok.
from autograd import grad, numpy as jnp
import numpy as np
def cast(params):
[A, B, C, L, M, N] = params
L = jnp.minimum(jnp.abs(L), 100)
M = jnp.minimum(jnp.abs(M), 100)
N = jnp.minimum(jnp.abs(N), 100)
return A, B, C, L, M, N
def pred(params, x):
[A, B, C, L, M, N] = cast(params)
return A *jnp.sin(2 * jnp.pi * L * x) + B*jnp.cos(2*jnp.pi * M * x) + C * jnp.sin(2 * jnp.pi * N * x)
x = [0,0.01,0.02,0.03,0.04,0.05,0.06,0.07,0.08,0.09,0.1,0.11,0.12,0.13,0.14,0.15,0.16,0.17,0.18,0.19,0.2,0.21,0.22,0.23,0.24,0.25,0.26,0.27,0.28,0.29,0.3,0.31,0.32,0.33,0.34,0.35,0.36,0.37,0.38,0.39,0.4,0.41,0.42,0.43,0.44,0.45,0.46,0.47,0.48,0.49,0.5,0.51,0.52,0.53,0.54,0.55,0.56,0.57,0.58,0.59,0.6,0.61,0.62,0.63,0.64,0.65,0.66,0.67,0.68,0.69,0.7,0.71,0.72,0.73,0.74,0.75,0.76,0.77,0.78,0.79,0.8,0.81,0.82,0.83,0.84,0.85,0.86,0.87,0.88,0.89,0.9,0.91,0.92,0.93,0.94,0.95,0.96,0.97,0.98,0.99]
y = [4,1.240062433,-0.7829654986,-1.332487982,-0.3337640721,1.618033989,3.512512389,4.341307895,3.515268061,1.118929599,-2.097886967,-4.990538967,-6.450324073,-5.831575611,-3.211486891,0.6180339887,4.425660706,6.980842552,7.493970785,5.891593744,2.824429495,-0.5926374511,-3.207870455,-4.263694544,-3.667432785,-2,-0.2617162175,0.5445886005,-0.169441247,-2.323237059,-5.175570505,-7.59471091,-8.488730333,-7.23200463,-3.924327772,0.6180339887,5.138501587,8.38127157,9.532377045,8.495765687,5.902113033,2.849529206,0.4768388529,-0.46697525,0.106795821,1.618033989,3.071952496,3.475795162,2.255463709,-0.4905371745,-4,-7.117914956,-8.727599664,-8.178077181,-5.544088451,-1.618033989,2.365340134,5.169257268,5.995297102,4.758922924,2.097886967,-0.8873135564,-3.06024109,-3.678989552,-2.666365632,-0.6180339887,1.452191817,2.529722611,2.016594378,-0.01374122059,-2.824429495,-5.285215072,-6.302694708,-5.246870619,-2.210419738,2,6.13956874,8.965976562,9.68000641,8.201089581,5.175570505,1.716858387,-1.02183483,-2.278560533,-1.953524751,-0.6180339887,0.7393509358,1.129293593,-0.02181188158,-2.617913164,-5.902113033,-8.727381729,-9.987404016,-9.043589913,-5.984648344,-1.618033989,2.805900027,6.034770001,7.255101454,6.368389697]
def loss(params):
p = pred(params, np.array(x))
return jnp.mean((np.array(y)-p)**2)
params = np.array([np.random.random()*100 for _ in range(6)])
for _ in range(10000):
g = grad(loss)
params = params - 0.001*g(params)
print("Relaxed solution", cast(params), "loss=", loss(params))
constrained_params = np.round(cast(params))
print("Integer solution", constrained_params, "loss=", loss(constrained_params))
print()
Since the problem will have a lot of local minima, you might need to run it multiple times.

It's quite hard to use gradient descent to find a solution to this problem, because it tends to get stuck when changing the L, M, or N parameters. The gradients for those can push it away from the right solution, unless it is very close to an optimal solution already.
There are ways to get around this, such as basinhopping or random search, but because of the function you're trying to learn, you have a better alternative.
Since you're trying to learn a sinusoid function, you can use an FFT to find the frequencies of the sine waves. Once you have those frequencies, you can find the amplitudes and phases used to generate the same sine wave.
Pardon the messiness of this code, this is my first time using an FFT.
import scipy.fft
import numpy as np
import math
import matplotlib.pyplot as plt
def get_top_frequencies(x, y, num_freqs):
x = np.array(x)
y = np.array(y)
# Find timestep (assume constant timestep)
dt = abs(x[0] - x[-1]) / (len(x) - 1)
# Take discrete FFT of y
spectral = scipy.fft.fft(y)
freq = scipy.fft.fftfreq(y.shape[0], d=dt)
# Cut off top half of frequencies. Assumes input signal is real, and not complex.
spectral = spectral[:int(spectral.shape[0] / 2)]
# Double amplitudes to correct for cutting off top half.
spectral *= 2
# Adjust amplitude by sampling timestep
spectral *= dt
# Get ampitudes for all frequencies. This is taking the magnitude of the complex number
spectral_amplitude = np.abs(spectral)
# Pick frequencies with highest amplitudes
highest_idx = np.argsort(spectral_amplitude)[::-1][:num_freqs]
# Find amplitude, frequency, and phase components of each term
highest_amplitude = spectral_amplitude[highest_idx]
highest_freq = freq[highest_idx]
highest_phase = np.angle(spectral[highest_idx]) / math.pi
# Convert it into a Python function
function = ["def func(x):", "return ("]
for i, components in enumerate(zip(highest_amplitude, highest_freq, highest_phase)):
amplitude, freq, phase = components
plus_sign = " +" if i != (num_freqs - 1) else ""
term = f"{amplitude:.2f} * math.cos(2 * math.pi * {freq:.2f} * x + math.pi * {phase:.2f}){plus_sign}"
function.append(" " + term)
function.append(")")
return "\n ".join(function)
x = [0,0.01,0.02,0.03,0.04,0.05,0.06,0.07,0.08,0.09,0.1,0.11,0.12,0.13,0.14,0.15,0.16,0.17,0.18,0.19,0.2,0.21,0.22,0.23,0.24,0.25,0.26,0.27,0.28,0.29,0.3,0.31,0.32,0.33,0.34,0.35,0.36,0.37,0.38,0.39,0.4,0.41,0.42,0.43,0.44,0.45,0.46,0.47,0.48,0.49,0.5,0.51,0.52,0.53,0.54,0.55,0.56,0.57,0.58,0.59,0.6,0.61,0.62,0.63,0.64,0.65,0.66,0.67,0.68,0.69,0.7,0.71,0.72,0.73,0.74,0.75,0.76,0.77,0.78,0.79,0.8,0.81,0.82,0.83,0.84,0.85,0.86,0.87,0.88,0.89,0.9,0.91,0.92,0.93,0.94,0.95,0.96,0.97,0.98,0.99]
y = [4,1.240062433,-0.7829654986,-1.332487982,-0.3337640721,1.618033989,3.512512389,4.341307895,3.515268061,1.118929599,-2.097886967,-4.990538967,-6.450324073,-5.831575611,-3.211486891,0.6180339887,4.425660706,6.980842552,7.493970785,5.891593744,2.824429495,-0.5926374511,-3.207870455,-4.263694544,-3.667432785,-2,-0.2617162175,0.5445886005,-0.169441247,-2.323237059,-5.175570505,-7.59471091,-8.488730333,-7.23200463,-3.924327772,0.6180339887,5.138501587,8.38127157,9.532377045,8.495765687,5.902113033,2.849529206,0.4768388529,-0.46697525,0.106795821,1.618033989,3.071952496,3.475795162,2.255463709,-0.4905371745,-4,-7.117914956,-8.727599664,-8.178077181,-5.544088451,-1.618033989,2.365340134,5.169257268,5.995297102,4.758922924,2.097886967,-0.8873135564,-3.06024109,-3.678989552,-2.666365632,-0.6180339887,1.452191817,2.529722611,2.016594378,-0.01374122059,-2.824429495,-5.285215072,-6.302694708,-5.246870619,-2.210419738,2,6.13956874,8.965976562,9.68000641,8.201089581,5.175570505,1.716858387,-1.02183483,-2.278560533,-1.953524751,-0.6180339887,0.7393509358,1.129293593,-0.02181188158,-2.617913164,-5.902113033,-8.727381729,-9.987404016,-9.043589913,-5.984648344,-1.618033989,2.805900027,6.034770001,7.255101454,6.368389697]
print(get_top_frequencies(x, y, 3))
That produces this function:
def func(x):
return (
5.00 * math.cos(2 * math.pi * 10.00 * x + math.pi * 0.50) +
4.00 * math.cos(2 * math.pi * 5.00 * x + math.pi * -0.00) +
2.00 * math.cos(2 * math.pi * 3.00 * x + math.pi * -0.50)
)
Which is not quite the format you specified - you asked for two sins and one cos, and for no phase parameter. However, using the trigonometric identity cos(x) = sin(pi/2 - x), you can convert this into an equivalent expression that matches what you want:
def func(x):
return (
5.00 * math.sin(2 * math.pi * -10.00 * x) +
4.00 * math.cos(2 * math.pi * 5.00 * x) +
2.00 * math.sin(2 * math.pi * 3.00 * x)
)
And there's the original function!

Finding a value in an equation

i`m looking for an easy way to find the value of a variable depending on the result of another one:
import numpy as np
import matplotlib.pyplot as plt
T = np.arange(0.01, 4.5, 0.0001)
N = (2.63 * 10 ** -16) * ((2.71828 ** (6.93 * T)) - 1) + ((4.05 * 10 ** -6) * T)
plt.plot(N,T)
plt.axis(xmin=-0.001, ymax=5)
plt.show()
For example I need the value of T for N= 0,00006762 (Or the closest value). This would be easy if I could solve for T, but I find it easier to create an array of the possible T`s and try the other way.

You can loop over values of T, calculate N, and compare it with the number you are looking for (0.00006762), and return the closest one you find:
target = 0.00006762
smallest_diff = 1000
best_answer = 'NA'
for T in np.arange(0.01, 4.5, 0.0001):
N = (2.63 * 10 ** -16) * ((2.71828 ** (6.93 * T)) - 1) + ((4.05 * 10 ** -6) * T)
if abs(N - target) < smallest_dif:
smallest_diff = abs(N - target)
best_answer = T
print(best_answer)

Fourier Series Summation in Python

I'm attempting to graph a Fourier series summation in Python. Thus far, I have this:
#! /usr/bin/env python
from sympy import *
import numpy
import matplotlib.pyplot as plt
n = Symbol('n')
x = Symbol('x')
L_1 = -1
L_2 = 1
f = -x
a_0 = (1 / L_2) * integrate(f, (x, L_1, L_2))
a_n = (1 / L_2) * integrate(f * cos(n * pi * x / L_2), (x, L_1, L_2))
b_n = (1 / L_2) * integrate(f * sin(n * pi * x / L_2), (x, L_1, L_2))
F = (a_0 / 2) + mpmath.fsum((a_n * cos(n * pi * x / L_2)) + (b_n * sin(n * pi * x / L_2)), [0, 20])
print(F)
However, fsum is throwing up an error:
F = (a_0 / 2) + mpmath.fsum((a_n * cos(n * pi * x / L_2)) + (b_n * sin(n * pi * x / L_2)), [0, 20])
File "C:\Program Files\Python\lib\site-packages\sympy\mpmath\ctx_mp_python.py", line 831, in fsum
for term in terms:
TypeError: 'Mul' object is not iterable
I'm not clear on what a Mul object is, or what my issue actually is. Does someone have any pointers for getting this summation working? Like I said, the end goal is to plug in some values for x and sum over n iterations, in this case 20.

I am not sure about why its throwing Mul object error, the error means the object is single item and not iterable.
Regarding the method you are trying to use mpmath.fsum, the signature is not matching with the documentation.
mpmath.fsum(terms, absolute=False, squared=False)
The method which you are using, seems match with nsum.
mpmath.nsum(ctx, f, *intervals, **options)
If you want to implement fourier transform in python, you can check the implementation here using python and numpy.

SymPy 1.0 has been included with fourier_series function that lets you compute Fourier Series of a function. Docs can be found here.
eg.
>>> f = fourier_series(-x, (x, -1, 1))
>>> f.truncate(5)
-2*sin(pi*x)/pi + sin(2*pi*x)/pi - 2*sin(3*pi*x)/(3*pi)

python numerically solving an equation with no sign change between the upper and lower bound

I am trying to solve an equation but I am unable to use brentq since there is no sign change. How could I find the value of r?
>>> import numpy as np
>>>
>>> def f(r):
return 0.1 + 1 / (2 * r ** 2) - 2 / (3 * np.sqrt(r ** 3))

I think there is sign change, but you can solve it by fsolve:
from scipy.optimize import fsolve
import numpy as np
def f(r):
return 0.1 + 1 / (2 * r ** 2) - 2 / (3 * np.sqrt(r ** 3))
r = fsolve(f, 1)
f(r)
result:
r = 2.22213541
Edit
There are two solution:
import numpy as np
def f(r):
return 0.1 + 1 / (2 * r ** 2) - 2 / (3 * np.sqrt(r ** 3))
r = np.linspace(0.6, 3, 1000)
plot(r, f(r))
output:

Matplotlib contour isn't working

I'm trying to plot the batman equation. A solution in sympy or matplotlib will be great (sage isn't cool because I'm using windows). The problem is that if I comment out certain parts the part of the figure appears but with all the F *= parts, I get a blank plot.
import matplotlib.pyplot
from numpy import arange
from numpy import meshgrid
from numpy import sqrt
from numpy import real
delta = 0.01
xrange = arange(-7.0, 7.0, delta)
yrange = arange(-3.0, 3.0, delta)
x, y = meshgrid(xrange,yrange)
F = 1
F *= (((x/7) ** 2) * sqrt(abs(abs(x) - 3)/(abs(x) - 3)) + ((y / 3) ** 2) * sqrt(abs(y + (3 * sqrt(33)) / 7)/(y + (3 * sqrt(33)) / 7)) - 1)
F *= (abs(x/2) - ((3 * sqrt(33) - 7)/112) * x**2 - 3 + sqrt(1 - (abs(abs(x) - 2) - 1) ** 2 ) - y)
F *= (9 * sqrt(abs((abs(x) - 1) * (abs(x) - 3/4))/((1 - abs(x)) * (abs(x) - 3/4))) - 8 * abs(x) - y)
F *= (3 * abs(x) + 0.75 * sqrt(abs((abs(x) - 3/4) * (abs(x) - 1/2))/((3/4 - abs(x)) * (abs(x) - 1/2))) - y)
F *= ((9/4) * sqrt(abs((x - 1/2) * (x + 1/2))/((1/2 - x) * (1/2 + x))) - y)
F *= ((6 * sqrt(10)) / 7 + (3/2 - abs(x)/2) * sqrt(abs(abs(x) - 1)/(abs(x) - 1)) - ((6 * sqrt(10))/ 14) * sqrt(4 - (abs(x) - 1) ** 2 ) - y)
G = 0
matplotlib.pyplot.contour(x, y, (F - G), [0])
matplotlib.pyplot.show()
What's going on here? If the graph is zero for one multiplicand, it should still be so no matter which other multiplicands I throw in there.
source of the batman equation: http://www.reddit.com/r/pics/comments/j2qjc/do_you_like_batman_do_you_like_math_my_math/

The parameter of sqrt is negative for many points, so the finally products are all NaN. You can plot every factor as following:
from __future__ import division # this is important, otherwise 1/2 will be 0
import matplotlib.pyplot
from numpy import arange
from numpy import meshgrid
from numpy import sqrt
from numpy import real
delta = 0.01
xrange = arange(-7.0, 7.0, delta)
yrange = arange(-3.0, 3.0, delta)
x, y = meshgrid(xrange,yrange)
F1 = (((x/7) ** 2) * sqrt(abs(abs(x) - 3)/(abs(x) - 3)) + ((y / 3) ** 2) * sqrt(abs(y + (3 * sqrt(33)) / 7)/(y + (3 * sqrt(33)) / 7)) - 1)
F2 = (abs(x/2) - ((3 * sqrt(33) - 7)/112) * x**2 - 3 + sqrt(1 - (abs(abs(x) - 2) - 1) ** 2 ) - y)
F3 = (9 * sqrt(abs((abs(x) - 1) * (abs(x) - 3/4))/((1 - abs(x)) * (abs(x) - 3/4))) - 8 * abs(x) - y)
F4 = (3 * abs(x) + 0.75 * sqrt(abs((abs(x) - 3/4) * (abs(x) - 1/2))/((3/4 - abs(x)) * (abs(x) - 1/2))) - y)
F5 = ((9/4) * sqrt(abs((x - 1/2) * (x + 1/2))/((1/2 - x) * (1/2 + x))) - y)
F6 = ((6 * sqrt(10)) / 7 + (3/2 - abs(x)/2) * sqrt(abs(abs(x) - 1)/(abs(x) - 1)) - ((6 * sqrt(10))/ 14) * sqrt(4 - (abs(x) - 1) ** 2 ) - y)
for f in [F1,F2,F3,F4,F5,F6]:
matplotlib.pyplot.contour(x, y, f, [0])
matplotlib.pyplot.show()
the result plot:

I know this might seem lame, but how about creating a list of x values, and then computing the value of "batman" at each of those positions, and storing in another list. You could define a function "batman" which computes the y value for each x value you pass in.
Then just plot those lists with matplotlib.
EDIT: Since you've made numpy arrays already to store the results, you could use those when computing the y values.

I'm not even sure how this equation would work, since I see divisions by zero arising in the first term (under the first square root, when abs(x) = 3), and imaginary numbers showing up in the last term (under the last square root, when {abs(x)-1}^2 > 4, ie x > 3 or x < -3).
What am I missing here? Is only the real part of the result used, and are divisions by zero ignored or approximated?
Running this, I do indeed see lots of RunTimeWarnings, and it is not unlikely matplotlib would get totally confused what numbers to work with (NaNs, Infs; trying print F at the end). Looks like it still manages when there's only a relatively low number of NaNs or Infs, which would explain that you're seeing part of the figure.
I'd think matplotlib's contour is fine, just confused by the input.