Hope I doesn't repeat any question, but couldn't find...
I'm trying to run a function with the same key parameter many times. I understand why the f function changes the x0 array, but I don't really understand why the g function takes every time different argument (y0 is constant).
I would be thankful if anyone can explain me this behaviour and give me a tip how to implement what I want (basically at the end I would like to have y == np.array([0, 30, 0]) ).
import numpy as np
x0 = np.zeros(3)
y0 = np.zeros(3)
def f(i, x = x0):
x[1] += i
return x
def g(i, y = y0.copy()):
print "y that goes to g (every time is different) \n", y
y[1] += i
return y
print "x0 before f \n" ,x0
x = f(5)
print "x0 after f is the same as x \n", x0, "\n", x
print "y0 before g \n" ,y0
for i in [10, 20, 30]:
y = g(i)
print "y0 after g doe not change, but y is NOT as I would expect! \n", y0, "\n", y
Default arguments to functions are evaluated only once, when the function is defined. This means that your function definition is equivalent to:
y0_ = y0.copy()
def g(i, y = y0_):
print "y that goes to g (every time is different) \n", y
etc
Which explains why your y argument changes every time.
"but I don't really understand why the g function takes every time different argument"
def g(i, y = y0.copy()):
....
Your y0 is constant but you are creating copy of y0 with different reference first time when g() function is called so you can't change y0 with function g(). Just change
y = y0.copy()
to
y=y0
Related
I am trying to fit 3-dimensional data (that is, 2 independent and 1 dependent variable) using multivariate fitting in scipy curve_fit. I wish to do piecewise fitting for the same problem. I have tried to proceed on the basis of this without any success. The problem is defined below:
import numpy as np
from scipy.optimize import curve_fit
#..........................................................................................................
def F0(X, a, b, c, c0, y0):
x, y = X
value = []
for i in range(0, len(x)):
if y[i] < y0:
lnZ = x[i] + c0*y[i]
else:
lnZ = x[i] + c*y[i]
val = a + (b*lnZ)
value.append(val)
return value
#..........................................................................................................
def F1(X, a, b, c):
x, y = X
lnZ = x + c*y
value = a + (b*lnZ)
return value
#..........................................................................................................
x = [-2.302585093,
-2.302585093,
-2.302585093,
-2.302585093,
-2.302585093,
-2.302585093,
-2.302585093,
0,
0,
0,
0,
0,
0,
0,
2.302585093,
2.302585093,
2.302585093,
2.302585093,
2.302585093,
2.302585093,
2.302585093
]
y = [7.55E-04,
7.85E-04,
8.17E-04,
8.52E-04,
8.90E-04,
9.32E-04,
9.77E-04,
7.55E-04,
7.85E-04,
8.17E-04,
8.52E-04,
8.90E-04,
9.32E-04,
9.77E-04,
7.55E-04,
7.85E-04,
8.17E-04,
8.52E-04,
8.90E-04,
9.32E-04,
9.77E-04
]
z = [4.077424497,
4.358253892,
4.610475878,
4.881769469,
5.153063061,
5.323277142,
5.462023074,
4.610475878,
4.840765517,
5.04864602,
5.235070966,
5.351407761,
5.440090728,
5.540693448,
4.960439843,
5.118257381,
5.266539115,
5.370479367,
5.440090728,
5.528296904,
5.5816974,
]
popt, pcov = curve_fit(F0, (x, y), z, method = 'lm')
print(popt)
popt, pcov = curve_fit(F1, (x, y), z, method = 'lm')
print(popt)
The output is:
[1.34957781e+00 1.05456428e-01 1.00000000e+00 4.14879613e+04
1.00000000e+00]
[1.34957771e+00 1.05456434e-01 4.14879603e+04]
You can see that the values of parameters in the piecewise fitting remain as the initial values. I know I am not doing it in the correct way. Please correct me.
The main source of the problem is the insensitivity of this approach to the value of the variable that defines the switch from one function to another (see this response for a similar explanation). Moreover, the choice of starting parameters isn't good.
Since no starting values are provided, curve_fit chooses a value of 1 for all the fitting parameters (see here the default value for p0). Since the fitting algorithm works by making small variations on the parameters, y0 is varied in small steps around 1, which produces no changes in the output of the function (all y values are much smaller than 1). Since y[i] < y0 is always True and only the first branch is ever evaluated, and the output of the function does not depend on the value of c. That explains why y0 and c stay at the initial values.
One might expect that setting y0 initial value to be inside of the range of values that are evaluated (i.e. around 8E-4) might solve the problem. Indeed, since the second branch is evaluated, the value of c is now optimized. Nevertheless, y0 value will stay unchanged. As the fitting algorithm works testing very small changes to the values, the changes are not large enough to move from the interval between two experimental y values to another one. In this particular case, if one chooses 8E-4, the small variations will never be enough to make it go over 8.17E-04 or below 7.85E-4, that are the values encompassing initial y0 choice.
One can usually circumvent this problem making the function depend explicitly on the value of y0. A smart choice would be to redefine the function so the value at y0 is the same no matter which branch is taken (i.e. ensure that the function is continuous). In this case, the function definition does not ensure so. A reasonable change would be:
def F2(X, a, b, c, c0, y0):
x, y = X
value = []
for i in range(0, len(x)):
lnZ = x[i] + c0 * y[i]
if y[i] >= y0:
lnZ += c * (y[i]-y0)
val = a + (b*lnZ)
value.append(val)
return value
which changes the meaning of the parameter c, and limits the results to only continuous functions. In this case, the value of y0 is indeed the function turning point. Nevertheless, it yields the desired results:
popt2, pcov = curve_fit(F2, (x, y), z, p0=(1, 1, 1E4, 1E4, 9.1E-4), method = 'lm')
print(popt2)
results in:
[-1.93417968e-01 1.05456433e-01 -3.65740192e+04 5.97890809e+04
8.64354057e-04]
A better (pythonic) definition for the function avoids the for loop:
def F3(X, a, b, c, c0, y0):
x, y = X
lnZ = x + c0 * y
idx = np.where(y>=y0)
lnZ[idx] += c * (y[idx] - y0)
rv = a + (b * lnZ)
return rv
which will probably be much faster for larger datasets.
I am trying to convert this code into MATLAB but I am not sure how to do the subscripts (Y[i] = Y[i-1]) as well as the func and f_exact variables
heres the code:
def Forward_Euler(y0,t0,T,dt,f):
t = np.arange(t0,T+dt,dt)
Y = np.zeros(len(t))
Y[0] = y0
for i in range(1,len(t)):
Y[i] = Y[i-1]+dt*f(Y[i-1], t[i-1])
return Y, t
func = lambda y,t: y-t
f_exact = lambda t: t+1-1/2*np.exp(t)
You can use anonymous functions in matlab:
func = #(y,t)(y - t)
f_exact = #(t)(t + 1 - exp(t)/2) % it works with any matrix t as well
And you can use for matrices as well (they should keep matrix operation rules). For example, in func function, as there is a minus in the form of function, the dimension of y and t must be the same.
My first py file is the function that I want to find the roots, like this:
def myfun(unknowns,a,b):
x = unknowns[0]
y = unknowns[1]
eq1 = a*y+b
eq2 = x**b
z = x*y + y/x
return eq1, eq2
And my second one is to find the value of x and y from a starting point, given the parameter value of a and b:
a = 3
b = 2
x0 = 1
y0 = 1
x, y = scipy.optimize.fsolve(myfun, (x0,y0), args= (a,b))
My question is: I actually need the value of z after plugging in the result of found x and y, and I don't want to repeat again z = x*y + y/x + ..., which in my real case it's a middle step variable without an explicit expression.
However, I cannot replace the last line of fun with return eq1, eq2, z, since fslove only find the roots of eq1 and eq2.
The only solution now is to rewrite this function and let it return z, and plug in x and y to get z.
Is there a good solution to this problem?
I believe that's the wrong approach. Since you have z as a direct function of x and y, then what you need is to retrieve those two values. In the listed case, it's easy enough: given b you can derive x as the inverse of eqn2; also given a, you can invert eqn1 to get y.
For clarity, I'm changing the names of your return variables:
ret1, ret2 = scipy.optimize.fsolve(myfun, (x0,y0), args= (a,b))
Now, invert the two functions:
# eq2 = x**b
x = ret2**(1/b)
# eq1 = a*y+b
y = (ret1 - b) / a
... and finally ...
z = x*y + y/x
Note that you should remove the z computation from your function, as it serves no purpose.
Let's say I have the following python code
y = 2
def f(x, y):
y = y**2
return x*y
for i in range(5):
print(f(2,y))
Is it somehow possible to make the change to y within f global while still passing it to f as an argument?
I know that
y = 2
def f(x, y):
global y
y = y**2
return x*y
for i in range(5):
print(f(2,y))
will not work because y cannot be both global and a function parameter.
The 'ugly solution that I have is simply not to pass y as an argument:
y = 2
def f(x):
global y
y = y**2
return x*y
for i in range(5):
print(f(2,y))
but I am not satisfied with this, as I would like to explicitly pass y to the function and basically call it by reference.
The background of this question is that I would like to use scipy's odeint, and I have to use sparse matrices in the computation of the derivative that also change with time.
If I want to avoid converting these to numpy and back to sparse at every timestep, I have to store them globally and modify them from within the function. Because the output of the function is dictated by odeint (it has to be said derivative) it is not an option to include these matrices in the output (and I don't know how that would work anyway, because I'd have to mix scalars and matrices in the output array).
It would be nice if I could somehow pass them as a parameter but make the changes to them from within the function globally permanent.
Just use a different name for the formal argument to f:
y = 2
def f(x, y2):
global y
y = y2**2
return x*y
for i in range(5):
print(f(2,y))
If I understand your intent, then I believe this should work for you.
You cannot do this exactly, for the reason you have described: a variable cannot be at the same time global and a local argument.
However, one solution would be to do this:
y_default = 2
def f(x, y=None):
if y is None:
y = y_default
y = y**2
return x*y
This will do what you want, as you can now call f(2) or f(2,3)
Essentially the problem is that y is global and local as the error message will suggest. Therefore you avoid the local variable issue by introducing a variable z locally. You can still pass y into z, which then yields the desired result.
y = 2
def f(x, z):
y = z**2
global y
return x*y
for i in range(5):
print f(2,y)
I am trying to define a function of n variables to fit to a data set. The function looks like this.
Kelly Function
I then want to find the optimal ai's and bj's to fit my data set using scipy.optimize.leastsq
Here's my code so far.
from scipy.optimize import leastsq
import numpy as np
def kellyFunc(a, b, x): #Function to fit.
top = 0
bot = 0
a = [a]
b = [b]
for i in range(len(a)):
top = top + a[i]*x**(2*i)
bot = bot + b[i]*x**(2*i)
return(top/bot)
def fitKelly(x, y, n):
line = lambda params, x : kellyFunc(params[0,:], params[1,:], x) #Lambda Function to minimize
error = lambda params, x, y : line(params, x) - y #Kelly - dataset
paramsInit = [[1 for x in range(n)] for y in range(2)] #define all ai and bi = 1 for initial guess
paramsFin, success = leastsq(error, paramsInit, args = (x,y)) #run leastsq optimization
#line of best fit
xx = np.linspace(x.min(), x.max(), 100)
yy = line(paramsFin, xx)
return(paramsFin, xx, yy)
At the moment it's giving me the error:
"IndexError: too many indices" because of the way I've defined my initial lambda function with params[0,:] and params[1,:].
There are a few problems with your approach that makes me write a full answer.
As for your specific question: leastsq doesn't really expect multidimensional arrays as parameter input. The documentation doesn't make this clear, but parameter inputs are flattened when passed to the objective function. You can verify this by using full functions instead of lambdas:
from scipy.optimize import leastsq
import numpy as np
def kellyFunc(a, b, x): #Function to fit.
top = 0
bot = 0
for i in range(len(a)):
top = top + a[i]*x**(2*i)
bot = bot + b[i]*x**(2*i)
return(top/bot)
def line(params,x):
print(repr(params)) # params is 1d!
params = params.reshape(2,-1) # need to reshape back
return kellyFunc(params[0,:], params[1,:], x)
def error(params,x,y):
print(repr(params)) # params is 1d!
return line(params, x) - y # pass it on, reshape in line()
def fitKelly(x, y, n):
#paramsInit = [[1 for x in range(n)] for y in range(2)] #define all ai and bi = 1 for initial guess
paramsInit = np.ones((n,2)) #better
paramsFin, success = leastsq(error, paramsInit, args = (x,y)) #run leastsq optimization
#line of best fit
xx = np.linspace(x.min(), x.max(), 100)
yy = line(paramsFin, xx)
return(paramsFin, xx, yy)
Now, as you see, the shape of the params array is (2*n,) instead of (2,n). By doing the re-reshape ourselves, your code (almost) works. Of course the print calls are only there to show you this fact; they are not needed for the code to run (and will produce bunch of needless output in each iteration).
See my other changes, related to other errors: you had a=[a] and b=[b] in your kellyFunc, for no good reason. This turned the input arrays into lists containing arrays, which made the next loop do something very different from what you intended.
Finally, the sneakiest error: you have input variables named x, y in fitKelly, then you use x and y is loop variables in a list comprehension. Please be aware that this only works as you expect it to in python 3; in python 2 the internal variables of list comprehensions actually leak outside the outer scope, overwriting your input variables named x and y.