my problem is the following: I have N different measurements of a quantity which depends on two other quantities that I also know. I would like to use find a function two variable function that approximates the data, and I thought that using Fourier transforms was a nice idea.
Does anybody has a suggestion on how should I proceed? I think as a first step I want to do a FFT of my data, but then how can I implement the inverse FT not only for the points where I measured but for any pair (x,y) as input?
Thanks a lot.
(I am using python).
Related
I'm new to Python but have this homework assignment I'm not prepared for. Please help.
I wrote:
import numpy as np
X,Y = np.loadtxt("wavedata.csv", usecols=(0, 1), delimiter=",", unpack=True)
and defined functions for a_0, a_n, b_n, and the trapezoid rule. But the CSV file only has from values from 0 to 10. Don't Fourier coefficients need integration from -L to L? And how would I use the trapezoid rule here, or make functions computing the coefficients?
Any tips? Any chunks of the program you can share with me? Thank you very much.
Not exactly clear what you are asking--perhaps a more specific framing of the question would be helpful.
However,I will try to answer. The Fourier coefficients are used to describe a periodic function as a sum of a fundamental frequency (which is defined by the period); but for a non-periodic data set, we take the period to be infinity (i.e. the pattern may repeat, but in an infinite time away), and that defines the Fourier transform.
So I think what you may be looking for is the coefficients of the Discrete Fourier Transform (DFT), which can be simply computed with numpy.fft.fft() (see their documentation for more details about how to use the function properly).
This question is regarding curve fitting in python.
First, I would say that I do not know the curve fit function to insert into "curve_fit" function in the scipy library; therefore, I am trying to use a polyfit which is OK if I am interested in interpolation but my goal is to predict values at future points, in other words extrapolation.
I have attached a screenshot of a raw signal, smoothed and its polyfit result. It has the correct poly order but still fails at extrapolation. My conclusion is that poly fit is not the right approach here, but I can not estimate the curve function. What are you thoughts?
Please note that this is not a distribution since the y values may keep slowly decreasing infinitely, even below 0.
I'd say the function looks like an exponential Gaussian but again it's not a distribution so dont want to do that.
My last thought was to split the plot into two, the first model can certainly be modeled as a polynomial and the second as an exponential. (values are different than first png cuz it's of a different signal).
Then, maybe combine the two. What do you think about this?
Attached is a screenshot of this too.
Since many curves can fit the data and extrapolate differently, you need to choose the right basis functions to get the behaviour you want.
So far you have tried polynomials for instance, these however tend to +- infinite, which is perhaps not what you want.
I would try and use curve_fit on a sum of Hermite polynomials or Laguerre polynomials. For instance, for Laguerre polynomials, you could try
a + b*exp(-k x) + c*(1-x)*exp(-k x) + d*(x^2 - 4*x + 2)*exp(-k x) + ...
Python has a lot of convenience functions built in for this, see e.g. https://docs.scipy.org/doc/numpy-1.13.0/reference/routines.polynomials.laguerre.html
Note however that you should also fit k to your data, which you could use curve_fit for.
I have multiple sets of data, each of which have a roughly sinusoidal shape. I have attached a photo of one such data Roughly sinusoidal data
I want to fit an equation of the form y = Asin(kx+b)+mx+c. Basically the data is a sine wave with a linear increase. Is there any simple way of doing this, preferably without toolboxes as I have about 20 of these data to sort through?
Thank you.
I am using matlab, however I am open to python too :).
What you want to do is to use regression in your function to fit the data.
One way to do this is to apply Non-linear least squares to find the parameters that minimize the square error for your function.
Take a look at the matlab function lsqcurvefit (http://es.mathworks.com/help/optim/ug/lsqcurvefit.html)
I need to calculate the area between two curves.
I have lots of data, so I'd like to do it programmatically.
Basically, I always have 2 normal distributions, calculated from a mean value and standard deviation. I would then like to calculate how much they intersect.
Here is an example of what I mean, and also some code in R (that I don't know).
Is there already a function in matplotlib or scipy or some other module that does it for me?
In case I have to implement it myself, I think that I should do:
find the intersections (there will be max 2)
see which function is lower before, [between], and after the intersection
calculate the integral of the lower function and add them all together
Is that right? How can I do the single steps? Are there functions, modules, etc that can help?
I don't know R either, but the answer seems to be in the link you provided: just integrate the minimum of your distributions.
You don't need to find intersections, just feed min(f(x), g(x)) to scipy.integrate.quad.
I have some data that are the integrals of an unknown curve within bins. For your interest, the data is ocean wave energy and the bins are for directions, e.g. 0-15 degrees. If possible, I would like to fit a curve on to the data that conserves the integrals within the bins. I've tried sketching it on a notepad with a pencil and it seems like it could be possible. Does anyone know of any curve-fitting tool in Python to do this, for example in the scipy interpolation sub-package?
Thanks in advance
Edit:
Thanks for the help. If I do it, it looks like I will try the method that is recommended in section 4 of this paper: http://journals.ametsoc.org/doi/abs/10.1175/1520-0485%281996%29026%3C0136%3ATIOFFI%3E2.0.CO%3B2. In theory, it basically uses matrices to make some 'fake' data from the known integrals between each band. When plotted, this data then produces an interpolated line graph that preserves the integrals.
It's a little outside my bailiwick, but I can suggest having a look at SciKits to see if there's anything there that might be useful. Other packages to browse would be pandas and StatsModels. Good luck!
If you have a curve f(x) which is an approximation to the integral of another curve g(x), i.e. f=int(g,x) then the two are related by the Fundamental theorem of calculus, that is, your original function is the derivative of the first curve g = df/dx. As such you can use numpy.diff or any of the higher order methods to approximate df/dx to obtain an estimate of your original curve.
One possibility: calculate the cumulative sum of the bin volumes (np.cumsum), fit an interpolating spline to it, and then take the derivative to get the curve.
scipy splines have methods to calculate the derivatives.
The only limitation, in case it is relevant in your case, the spline through the cumulative sum might not be monotonic, and the derivative might be negative over some intervals.
I guess that the literature on smoothing a histogram looks at similar constraints on the volume of the integral/bin, but I don't have any references ready.
1/ fit2histogram
Your question is about fitting an histogram. I just came through documentation for some Python package for Multi-Variate Pattern Analysis, PyMVPA, and some function for histogram fitting is proposed. An example is here: PyMVPA.
However, I guess that set of available distributions is limited to famous distributions.
2/ integral computation
As already mentionned, next solution is to approximate integral value, and to fit a model to the resulting set of data. Either you know explicit expression for the derivative, or you use computational derivation: finite difference, analytical method.