Develop Reference

Problems using numpy.piecewise

1. The core problem and question
I will provide an executable example below, but let me first walk you through the problem first.
I am using solve_ivp from scipy.integrate to solve an initial value problem (see documentation). In fact I have to call the solver twice, to once integrate forward and once backward in time. (I would have to go unnecessarily deep into my concrete problem to explain why this is necessary, but please trust me here--it is!)
sol0 = solve_ivp(rhs,[0,-1e8],y0,rtol=10e-12,atol=10e-12,dense_output=True)
sol1 = solve_ivp(rhs,[0, 1e8],y0,rtol=10e-12,atol=10e-12,dense_output=True)
Here rhs is the right hand side function of the initial value problem y(t) = rhs(t,y). In my case, y has six components y[0] to y[5]. y0=y(0) is the initial condition. [0,±1e8] are the respective integration ranges, one forward and the other backward in time. rtol and atol are tolerances.
Importantly, you see that I flagged dense_output=True, which means that the solver does not only return the solutions on the numerical grids, but also as interpolation functions sol0.sol(t) and sol1.sol(t).
My main goal now is to define a piecewise function, say sol(t) which takes the value sol0.sol(t) for t<0 and the value sol1.sol(t) for t>=0. So the main question is: How do I do that?
I thought that numpy.piecewise should be tool of choice to do this for me. But I am having trouble using it, as you will see below, where I show you what I tried so far.
2. Example code
The code in the box below solves the initial value problem of my example. Most of the code is the definition of the rhs function, the details of which are not important to the question.
import numpy as np
from scipy.integrate import solve_ivp
# aux definitions and constants
sin=np.sin; cos=np.cos; tan=np.tan; sqrt=np.sqrt; pi=np.pi;
c = 299792458
Gm = 5.655090674872875e26
# define right hand side function of initial value problem, y'(t) = rhs(t,y)
def rhs(t,y):
p,e,i,Om,om,f = y
sinf=np.sin(f); cosf=np.cos(f); Q=sqrt(p/Gm); opecf=1+e*cosf;
R = Gm**2/(c**2*p**3)*opecf**2*(3*(e**2 + 1) + 2*e*cosf - 4*e**2*cosf**2)
S = Gm**2/(c**2*p**3)*4*opecf**3*e*sinf
rhs = np.zeros(6)
rhs[0] = 2*sqrt(p**3/Gm)/opecf*S
rhs[1] = Q*(sinf*R + (2*cosf + e*(1 + cosf**2))/opecf*S)
rhs[2] = 0
rhs[3] = 0
rhs[4] = Q/e*(-cosf*R + (2 + e*cosf)/opecf*sinf*S)
rhs[5] = sqrt(Gm/p**3)*opecf**2 + Q/e*(cosf*R - (2 + e*cosf)/opecf*sinf*S)
return rhs
# define initial values, y0
y0=[3.3578528933149297e13,0.8846,2.34921,3.98284,1.15715,0]
# integrate twice from t = 0, once backward in time (sol0) and once forward in time (sol1)
sol0 = solve_ivp(rhs,[0,-1e8],y0,rtol=10e-12,atol=10e-12,dense_output=True)
sol1 = solve_ivp(rhs,[0, 1e8],y0,rtol=10e-12,atol=10e-12,dense_output=True)
The solution functions can be addressed from here by sol0.sol and sol1.sol respectively. As an example, let's plot the 4th component:
from matplotlib import pyplot as plt
t0 = np.linspace(-1,0,500)*1e8
t1 = np.linspace( 0,1,500)*1e8
plt.plot(t0,sol0.sol(t0)[4])
plt.plot(t1,sol1.sol(t1)[4])
plt.title('plot 1')
plt.show()
3. Failing attempts to build piecewise function
3.1 Build vector valued piecewise function directly out of sol0.sol and sol1.sol
def sol(t): return np.piecewise(t,[t<0,t>=0],[sol0.sol,sol1.sol])
t = np.linspace(-1,1,1000)*1e8
print(sol(t))
This leads to the following error in piecewise in line 628 of .../numpy/lib/function_base.py:
TypeError: NumPy boolean array indexing assignment requires a 0 or 1-dimensional input, input has 2 dimensions
I am not sure, but I do think this is because of the following: In the documentation of piecewise it says about the third argument:
funclistlist of callables, f(x,*args,**kw), or scalars
[...]. It should take a 1d array as input and give an 1d array or a scalar value as output. [...].
I suppose the problem is, that the solution in my case has six components. Hence, evaluated on a time grid the output would be a 2d array. Can someone confirm, that this is indeed the problem? Since I think this really limits the usefulness of piecewiseby a lot.
3.2 Try the same, but just for one component (e.g. for the 4th)
def sol4(t): return np.piecewise(t,[t<0,t>=0],[sol0.sol(t)[4],sol1.sol(t)[4]])
t = np.linspace(-1,1,1000)*1e8
print(sol4(t))
This results in this error in line 624 of the same file as above:
ValueError: NumPy boolean array indexing assignment cannot assign 1000 input values to the 500 output values where the mask is true
Contrary to the previous error, unfortunately here I have so far no idea why it is not working.
3.3 Similar attempt, however first defining functions for the 4th components
def sol40(t): return sol0.sol(t)[4]
def sol41(t): return sol1.sol(t)[4]
def sol4(t): return np.piecewise(t,[t<0,t>=0],[sol40,sol41])
t = np.linspace(-1,1,1000)
plt.plot(t,sol4(t))
plt.title('plot 2')
plt.show()
Now this does not result in an error, and I can produce a plot, however this plot doesn't look like it should. It should look like plot 1 above. Also here, I so far have no clue what is going on.
Am thankful for help!

You can take a look to numpy.piecewise source code. There is nothing special in this function so I suggest to do everything manually.
def sol(t):
ans = np.empty((6, len(t)))
ans[:, t<0] = sol0.sol(t[t<0])
ans[:, t>=0] = sol1.sol(t[t>=0])
return ans
Regarding your failed attempts. Yes, piecewise excpect functions return 1d array. Your second attempt failed because documentation says that funclist argument should be list of functions or scalars but you send the list of arrays. Contrary to the documentation it works even with arrays, you just should use the arrays of the same size as t < 0 and t >= 0 like:
def sol4(t): return np.piecewise(t,[t<0,t>=0],[sol0.sol(t[t<0])[4],sol1.sol(t[t>=0])[4]])

How to resolve TypingErrors when the property of an ak.Array subclass relies on Numba-compiled functions?

I've perused the docs and demo notebooks in the awkward1 repo (and it's entirely possible I missed something obvious), but I've strayed into unfamiliar territory and would like some help.
Suppose I have points in a polar coordinate frame that I provide to users as an array of records using the awkward1 library. Some users may prefer to work with the points in a Cartesian coordinate frame, so I can accommodate them by adding behaviors, and the following example works.
import awkward1 as ak
import numpy as np
class Polar2Cartesian(object):
"""Mixin class to convert from polar to Cartesian."""
#property
def x(self):
return self.r * np.cos(self.theta)
# And similarly for y, but this is just an example
# Register the behaviors with awkward
class Point(ak.Record, Polar2Cartesian):
pass
class PointArray(ak.Array, Polar2Cartesian):
pass
ak.behavior['Point'] = Point
ak.behavior['*', 'Point'] = PointArray
# Define the points
r = np.array([2.53, 0.29, 3.18])
theta = np.array([np.pi/4, np.pi, -np.pi/3])
points = ak.zip({'r': r, 'theta': theta}, with_name='Point')
# x-component of the first point
points[0].x == r[0] * np.cos(theta[0]) # True
# x-components of all points
points.x == r * np.cos(theta) # <Array [True, True, True] type='3 * bool'>
Now suppose the conversion must use a Numba-compiled function. I've defined one in the following example, but in principle it could come from a third-party library over which I have no control. The following works for individual records, but raises a TypingError over the array of records.
import awkward1 as ak
import numba as nb
import numpy as np
#nb.njit
def cos(x):
"""A contrived example appears."""
return np.cos(x)
class Polar2Cartesian(object):
"""Mixin class to convert from polar to Cartesian."""
#property
def x(self):
return self.r * cos(self.theta) # uses jitted function here
# And similarly for y, but this is just an example
# Define and register the behaviors like before, and create
# the same array of Points as in the previous example
# This still works
points[0].x == r[0] * np.cos(theta[0]) # True
# This now raises a TypingError
points.x == r * np.cos(theta)
Parsing the traceback reveals the reason for the TypingError:
TypingError: can't resolve ufunc cos for types (awkward1.ArrayView(awkward1.NumpyArrayType(array(float64, 1d, A), none, {}), None, ()),)
That's fair, as it would be unreasonable to expect Numba to know how to use this type without help from the developer.
A workaround I have found is to call np.asarray() within the definition of Polar2Cartesian.x, i.e. cos(np.asarray(self.theta)). Although cos returns a NumPy array, Polar2Cartesian.x ends up becoming an awkward Array due to the subsequent multiplication with self.r. In general, one could convert the return value to an awkward record or array as needed.
But is this solution "awkward1-approved", or should I be going the route where I provide typer and lower functions for theta and x as additional behaviors? If the latter, could someone walk me through how to correctly write the typer and lower functions?
At this point, I wonder if the scope of my question title is too broad. If someone has a better suggestion, please let me know.

Piecewise Function lmfit

I am trying to define a piecewise function to be fitted by lmfit library in Python. The issue I am having is a parameter I have defined for the function will not evaluate alongside the data I am submitting.
I have one example of a case somewhat similar to mine here. However, the vectorize function the answer describes wasn't producing values I wanted, and when reading the documentation, it didn't seem to be the answer to my solution. I also used scipy.optimize.leastsq, but I got the same issue with lmfit described below.
I have a my residual function defined such as
from lmfit import minimize, Parameters, Model
def residual(params, y, x):
param1 = params['one']
param2 = params['two']
if(param2 < x):
p = 1
else:
p = param1*x + param2
return p - y
params = Parameters()
params.add('one', value=1)
params.add('two', value=2)
out = minimize(residual, params,args=(y,x))
I also tried defining the function such that
def f(param1,param2,x):
if(param2 < x):
p = 1
else:
p = param1*x + param2
return p
def residual(params, y, x):
param1 = params['one']
param2 = params['two']
return f(param1,param2,x) - y
I have also tried inline using a lambda function.
I am getting an error 'The truth value of an array with more than one element is ambiguous.' When I got the error, it made sense why it happened, because (param2 < x) would produce a logical array. However, I can't seem to find a way to define the function in a piecewise fashion with the given case to get it fitted with the lmfit.minimize() function. I have seen the answer done in Matlab, in which it's nlinfit function seems to evaluate the data element-wise without issue (I tried searching if Python has an equivalent operation to define element-wise computation such as .* or .+, but that doesn't seem to exist as explicitly).
lmfit also seems to operate a bit differently compared to nlinfit, because we have to always have our residuals return (model - y) while nlinfit outputs the result once the function is given, which I am not sure could be another issue.
So to reiterate, my main question is if there is a method of defining the piecewise function such that it can compare the parameter to the data set.
Any help or explanation would be appreciated, thank you!

In place of (param2 < x) (where param2 is a float and x is an numpy array), you want to use numpy.where. You might try:
def residual(params, y, x):
param1 = params['one']
param2 = params['two']
p = param1 * x + param2
p[np.where(param2 < x)] = 1.0
return p - y
I should also warn you about a potential problem with this approach to having a variable be a boundary for a piecewise function.
In non-linear fits, variables are always floating point (continuous, non-discrete) values. As the fit proceeds, it will make small adjustments in the values and see how that small change alters the result. In your approach, the parameter 'two' is used as both the transition between pieces and the offset for the line -- that is good.
If a parameter is used only as the transition, it may not work. Consider, say, x=np.array([0, 1., 2., 3., 4., ..., 20.0]). Having two = 10.5 and two=10.4 would then give the same result. In that case, the fit would not be able to alter the value of two: it would try a very small change, see no change in the result and give up.
So, either make sure that two is also used elsewhere in your real model (assuming your real model is more complicated than the example given), or consider using a more gentle transition rather than a hard change in pieces. I find an error-function of width ~spacing between x points often works. Depending on the nature of your problem, you might try something like this:
from scipy.special import erf, erfc
def residual(params, y, x):
param1 = params['one']
param2 = params['two']
dx = (max(x) - min(x))/(len(x)-1)
xhi = (erf((x-param2)/dx) + 1)/2.0
xlo = (erfc((x-param2)/dx) + 1)/2.0
p = xlo*1.0 + xhi*(param1*x + param2)
# note: did you really want?
# p = xlo*param + xhi*(param1*x + param2)
# p = param2 + xhi*param1*x
return p - y
Hope that helps.

Pass args for solve_ivp (new SciPy ODE API)

For solving simple ODEs using SciPy, I used to use the odeint function, with form:
scipy.integrate.odeint(func, y0, t, args=(), Dfun=None, col_deriv=0, full_output=0, ml=None, mu=None, rtol=None, atol=None, tcrit=None, h0=0.0, hmax=0.0, hmin=0.0, ixpr=0, mxstep=0, mxhnil=0, mxordn=12, mxords=5, printmessg=0)[source]
where a simple function to be integrated could include additional arguments of the form:
def dy_dt(t, y, arg1, arg2):
# processing code here
In SciPy 1.0, it seems the ode and odeint funcs have been replaced by a newer solve_ivp method.
scipy.integrate.solve_ivp(fun, t_span, y0, method='RK45', t_eval=None, dense_output=False, events=None, vectorized=False, **options)
However, this doesn't seem to offer an args parameter, nor any indication in the documentation as to implementing the passing of args.
Therefore, I wonder if arg passing is possible with the new API, or is this a feature that has yet to be added? (It would seem an oversight to me if this features has been intentionally removed?)
Reference:
https://docs.scipy.org/doc/scipy/reference/integrate.html

Relatively recently there appeared a similar question on scipy's github. Their solution is to use lambda:
solve_ivp(fun=lambda t, y: fun(t, y, *args), ...)
And they argue that there is already enough overhead for this not to matter.

It doesn't seem like the new function has an args parameter. As a workaround you can create a wrapper like
def wrapper(t, y):
orig_func(t,y,hardcoded_args)
and pass that in.

Recently the 'args' option was added to solve_ivp, see here: https://github.com/scipy/scipy/issues/8352#issuecomment-535689344

According to Javier-Acuna's ultra-brief, ultra-useful answer, the feature that you (as well as I) desire has recently been added. This was announced on Github by none other than the great Warren Weckesser (See his Github, StackOverflow) himself.
Anyway, jokes aside the docstring of solve_ivp has an example using it in for the `Lotka-Volterra equations
solve_ivp(
fun,
t_span,
y0,
method='RK45',
t_eval=None,
dense_output=False,
events=None,
vectorized=False,
args=None,
**options,
)
So, just include args as a tuple. In your case
args = (arg1, arg2)
Please don't use my answer unless your scipy version >= 1.4 .
There is no args parameter in solve_ivp for versions below it. I have personally experienced my answer failing for version 1.2.1.
The implementation by zahabaz would probably still work fine in case your scipy version < 1.4

For completeness, I think you can also do this but I'm not sure why you would bother since the other two options posted here are perfectly fine.
from functools import partial
fun = partial(dy_dt, arg1=arg1, arg2=arg2)
scipy.integrate.solve_ivp(fun, t_span, y0, method='RK45', t_eval=None, dense_output=False, events=None, vectorized=False, **options)

Adding to Cleb's answer, here's an example for using the lambda t,y: fun(t,y,args) method. We set up the function handle that returns the rhs of a second order homogeneous ODE with two parameters. Then we feed it to our solver, along with a couple options.
import numpy as np
from scipy import integrate
import matplotlib.pyplot as plt
def rhs_2nd_order_ode(t, y, a, b):
"""
2nd order ODE function handle for use with scipy.integrate.solve_ivp
Solves u'' + au'+ bu = 0 after reducing order with y[0]=u and y[1]=u'.
:param t: dependent variable
:param y: independent variables
:param a: a
:param b: b
:return: Returns the rhs of y[0]' = y[1] and y[1]' = -a*y[1] - b*y[0]
"""
return [y[1], -a*y[1] - b*y[0]]
if __name__ == "__main__":
t_span = (0, 10)
t_eval = np.linspace(t_span[0], t_span[1], 100)
y0 = [0, 1]
a = 1
b = 2
sol = integrate.solve_ivp(lambda t,y: rhs_2nd_order_ode(t,y,a,b), t_span, y0,
method='RK45', t_eval=t_eval)
fig, ax = plt.subplots(1, 1)
ax.plot(sol.t, sol.y[0])
ax.set(xlabel='t',ylabel='y')

How to call function for scipy.optimize.fmin_cg(func) in Python

I will simply explain the problem in short. This problem is exactly similar as shown in scipy.doc. The problem is on error occurance as float argument required, not numpy.ndarray
What I have:
Function: y = s*z^t
Variable length/dimensions
t - 1...m,
s - 1...m and 1...n. So, m is row number, n - col number.
z - 1...n.
y - this can be y1, y[2], y[3],..., y[m],
T - s[m,n] matrix
Like this:
y[1] = s[1][1]*z[1]^t[1]+s[1][2]*z[2]^t[1]+...s[1][n]*z[n]^t[1])
y[2] = s[2][1]*z[1]^t[2]+s[2][2]*z[2]^t[2]+...s[2][n]*z[n]^t[2])
...
y[m] = s[m][1]*z[1]^t[m]+s[m][2]*z[2]^t[2]+...s[m][n]*z[n]^t[m])
Problem: Error occured.
Optimization terminated successfully.
Traceback (most recent call last):
solution = optimize.fmin_cg(func, z, fprime=gradf, args=args)
File "C:\Python27\lib\site-packages\scipy\optimize\optimize.py", line 952, in fmin_cg
res = _minimize_cg(f, x0, args, fprime, callback=callback, **opts)
File "C:\Python27\lib\site-packages\scipy\optimize\optimize.py", line 1072, in _minimize_cg
print " Current function value: %f" % fval
TypeError: float argument required, not numpy.ndarray
Here is the code
import numpy as np
import scipy as sp
import scipy.optimize as optimize
def func(z, *args):
y,T,t = args[0]
return y - counter(T,z,t)
def counter(T, z, t):
rows,cols = np.shape(T)
res = np.zeros(rows)
for i,row_val in enumerate(T):
res[i] = np.dot(row_val, z**t[i])
return res
def gradf(z, *args):
y,T,t = args[0]
return np.dot(t,counter(T,z,t-1))
def main():
# Inputs
N = 30
M = 20
z0 = np.zeros(N) # initial guess
y = 30*np.random.random(M)
T = 10*np.random.random((M,N))
t = 5*np.random.random(M)
args = [y, T, t]
solution = optimize.fmin_cg(func, z0, fprime=gradf, args=args)
print 'solution: ', solution
if __name__ == '__main__':
main()
I also tried to find similar examples but could not find something very similar. Here is the code for your consideration. Thanks in advance.

The root of your problem is that fmin_cg expects the function to return a single scalar value for the misfit instead of an array.
Basically, you want something vaguely similar to:
def func(z, y, T, t):
return np.linalg.norm(y - counter(T,z,t))
I'm using np.linalg.norm here because there's no builtin function in numpy for the root-mean-square. The actual RMS would be norm(x) / sqrt(x.size), but for minimization the constant multiplier doesn't make any difference.
There are also other minor problems in your code (e.g. args[0] is going to be a single item. You want y, T, t = args or better yet, just func(z, y, T, t)). Your gradient function doesn't make any sense to me, but it's optional regardless. Also, there's no way the solution can produce reasonable values at the moment, as you're testing it against pure noise. I assume those are just meant to be placeholder values, though.
However, you have a larger problem. You're trying to minimize in 30-dimensional space. Most non-linear solvers aren't going to work well with that high of a dimensionality. It may work fine, but you're very likely to run into problems.
All that having been said, you may find it more intuitive to use the scipy.optimize.curve_fit interface rather than using the others, if you're okay with LM instead of CG (they're fairly similar methods).
One final thing: You're trying to solve for 30 model parameters with 20 observations. This is an underdetermined problem. This problem doesn't have a unique solution. You're going to need to apply some a-priori knowledge to get a reasonable answer.