Develop Reference

Issue with vectorization in python

I have an application where I need to merge two solutions obtained from the scipy.integrate.solve_ivp in python. The actual application is a bit more complicated, but the following example shows the idea:
from scipy.integrate import solve_ivp import numpy as np
def lotkavolterra(t, z, a, b, c, d):
x, y = z
return [a*x - b*x*y, -c*y + d*x*y]
sol_1 = solve_ivp(lotkavolterra, [0, 10], [10, 5], args=(1.5, 1, 3, 1), dense_output=True).sol
sol_2 = solve_ivp(lotkavolterra, [10, 15], [10, 5], args=(1.5, 1, 3, 1), dense_output=True).sol
def sol_comb(t):
if t <= 10:
return sol_1(t)
else:
return sol_2(t)
I want to be able to use the merged or combined solution sol_comb on numpy arrays. Hence I tried to define a vectorized solution as follows:
sol_comb_vect = np.vectorize(sol_comb)
The following code, where I only call the functions on scalars, works fine:
print("sol_1 for t = 2",sol_1(2))
print("sol_2 for t = 11",sol_2(11))
print("sol_comb for t = 11",sol_comb(11))
print("sol_comb_vect for t = 11",sol_comb_vect(11))
The individual solutions sol_1 and sol_2 are apparently vectorized, since the following works fine:
print("sol_1 for t = [2,3]",sol_1(np.array([2])))
print("sol_2 for t = [11,13]",sol_2(np.array([11,13])))
However, if I call the non-vectorized function sol_comb on an array, as in the following example, I get the expected ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all():
print("sol_comb for t = [11,13]",sol_comb(np.array([11,13])))
I was expecting the vectorized version sol_comb_vect to work. However, in the following, I get the error ValueError: setting an array element with a sequence.
print("sol_comb_vect for t = [11,13]",sol_comb_vect(np.array([11,13])))
Any ideas how to fix this?
I would also be happy to merge the two OdeSolution instances in a cleaner way. In principle I think this should be possible, by using the time values and interpolants for sol_1 and sol_2, respectively.

I think you need to specify the signature for your output when you vectorize your function since by default the pyfunc you pass to np.vectorize() is assumed to take scalars as input and output see doc. And I assume that your ValueError is caused by that. So try this:
sol_comb_vect = np.vectorize(sol_comb, signature='()->(n)')
sol_comb_vect(np.array([2, 11, 13]))
output:
array([[0.60031288, 0.09618044],
[0.21298705, 1.36999868],
[2.58274789, 0.01857732]])
I don't know if this is the expected output tho. I hope this answers your question.

Getting error saying the truth value of an array with more than one element is ambiguous

...and a suggestion to Use a.any() or a.all().
I am new to python and i am trying to implement a sabr model. I have defined a function with the following parameters:
def haganimpliedvol(a,f,k,B,v,t,p):
if k != f:
z = v/a*math.pow(f*k,(1-B)/2)*math.log(f/k)
xz = math.log((math.sqrt(1-2*p*z+math.pow(z,2))+z-p)/(1-p))
sigma = a/math.pow(f*k,(1-B)/2)*(1 + math.pow(1-B,2)/24* math.pow(math.log(f/k),2)+\
math.pow(1-B,4)/1920* math.pow(math.log(f/k),4))*\
xz*\
(1+(math.pow(1-B,2)/24*math.pow(a,2)/math.pow(f/k,1-B)+1/4*(p*B*v*a)/math.pow(f/k,(1-B)/2)+\
(2-3*math.pow(p,2))/24*math.pow(v,2)))*t
else:
sigma = a/math.pow(f,1-B)*\
(1+(math.pow(1-B,2)/24*math.pow(a,2)/math.pow(f,(2-2*B))+\
1/4*(p*B*a*v)/math.pow(f,1-B)+(2-3*math.pow(p,2))/24*math.pow(v,2)))*t
return(sigma)
Now I define another function to and call the haganimpliedvol() function
params = [0.4,0.6,0.1,-0.4]
def objective(params):
global k,sigma_iv,t,f
a = params[0]
B = params[1]
v = params[2]
p = params[1]
for (i,j,k) in zip(k,t,f):
calc_vols = np.array([haganimpliedvol(a,f,k,B,v,t,p)])
return(calc_vols)
As can be seen, a few parameters in the functions are list. I want to get an array as an output. However, I keep getting the message in the subject line.

Pay attention to the variables in this call:
for (i,j,k) in zip(k,t,f):
calc_vols = np.array([haganimpliedvol(a,f,k,B,v,t,p)])
for the zip to work, k,t, f have to be lists or arrays of matching size;
Done use k for an iteration variable; it is already used in the zip. I think you are just being careless here; or confused.
And the arguments to the hagen... function. Are the f, k, t supposed to be variables used in the zip? It would make more sense to use the iteration variables (i,j,?). Again, this just looks like you are careless, or don't care what happens.
As for the ambiguity error, that most likely arises in the
if k != f:
If either k or f is an array (or both) the k!=f will be a boolean array. That can't be used in if, which requires a simple True or False value. It does not iterate on the conditions. It is a basic Python if - a switch.
This ambiguity error comes up frequently, in various contexts, but all with the same basic issue - using an array in a context that requires a scalar T/F. A simple web search should provide lots of examples.

#hpaulj thank you for leading me on the right path. I vectorized my function and made some edits and now it is working fine.
haganimpliedvol = np.vectorize(haganimpliedvol,excluded = ['a','B','v','p'])
params = [0.2,0.7,0.01,-0.4]
def objective(params):
global k,sigma_iv,t,f
a = params[0]
B = params[1]
v = params[2]
p = params[1]
calc_vols = haganimpliedvol(a,f,k,B,v,t,p)
return(calc_vols)

Are you sure you want to pass arrays into the haganimpliedvol() function?
The general convention is to write functions which take a single input type.
Maybe call it one per item in the array?
Or write the function in a way that, if it sees the input is a list it iterates and if it sees the inputs arent lists then it just calculates it one time.
See this thread for ideas
How to make a function that can handle single inputs or lists of inputs

Insert newline after equals sign in self documenting f-string in python3.8

With python3.8, a new feature is self documenting format strings. Where one would normally do this:
>>> x = 10.583005244
>>> print(f"x={x}")
x=10.583005244
>>>
One can now do this, with less repetition:
>>> x = 10.583005244
>>> print(f"{x=}")
x=10.583005244
>>>
This works very well for one line string representations. But consider the following scenario:
>>> import numpy as np
>>> some_fairly_long_named_arr = np.random.rand(4,2)
>>> print(f"{some_fairly_long_named_arr=}")
some_fairly_long_named_arr=array([[0.05281443, 0.06559171],
[0.13017109, 0.69505908],
[0.60807431, 0.58159127],
[0.92113252, 0.4950851 ]])
>>>
Here, the first line does not get aligned, which is (arguably) not desirable. I would rather prefer the output of the following:
>>> import numpy as np
>>> some_fairly_long_named_arr = np.random.rand(4,2)
>>> print(f"some_fairly_long_named_arr=\n{some_fairly_long_named_arr!r}")
some_fairly_long_named_arr=
array([[0.06278696, 0.04521056],
[0.33805303, 0.17155518],
[0.9228059 , 0.58935207],
[0.80180669, 0.54939958]])
>>>
Here, the first line of the output is aligned as well, but it defeats the purpose of not repeating the variable name twice in the print statement.
The example is a numpy array, but it could have been a pandas dataframe etc. as well.
Hence, my question is: Can a newline character be inserted after the = sign in self documenting strings?
I tried to add it like this, but it does not work:
>>> print(f"{some_fairly_long_named_arr=\n}")
SyntaxError: f-string expression part cannot include a backslash
I read the docs on format-specification-mini-language, but most of the formatting there only works for simple data types like integers, and I was not able to achieve what I wanted using those that work.
Sorry for the long write-up.

Wouldn't recommend this at all, but for possibility's sake:
import numpy as np
_old_array2string = np.core.arrayprint._array2string
def _array2_nice_string(*args, **kwargs):
non_nice_string = _old_array2string(*args, **kwargs)
dimension_strings = non_nice_string.split("\n")
if len(dimension_strings) > 1:
dimension_string = dimension_strings[1]
dimension_indent = len(dimension_string) - len(dimension_string.lstrip())
return "\n" + " " * dimension_indent + non_nice_string
return non_nice_string
np.core.arrayprint._array2string = _array2_nice_string
Outputs for:
some_fairly_long_named_arr = np.random.rand(2, 2)
print(f"{some_fairly_long_named_arr=}")
some_fairly_long_named_arr=array(
[[0.95900608, 0.79367873],
[0.58616975, 0.17757661]])
and
some_fairly_long_named_arr = np.random.rand(1, 2)
print(f"{some_fairly_long_named_arr=}")
some_fairly_long_named_arr=array([[0.62492772, 0.80453153]]).
I made it so if if the first dimension is 1, it is kept on the same line.
There is a non-internal method np.array2string that I tried to re-assign, but I never got that working. If someone could find a way to re-assign that public function instead of this internally used one, I'd imagine that'd make this solution a lot cleaner.

I figured out a way to accomplish what I wanted, after reading through the CPython source:
import numpy as np
some_fairly_long_named_arr = np.random.rand(4, 2)
print(f"""{some_fairly_long_named_arr =
}""")
Which produces:
some_fairly_long_named_arr =
array([[0.23560777, 0.96297907],
[0.18882751, 0.40712246],
[0.61351814, 0.1981144 ],
[0.27115495, 0.72303859]])
I would rather prefer a solution that worked in a single line, but this seems to be the only way for now. Perhaps another way will be implemented in a later python version.
However note that the indentation on the continuation line has to be removed for the above mentioned method, as such:
# ...some code with indentation...
print(f"""{some_fairly_long_named_arr =
}""")
# ...more code with indentation...
Otherwise, the alignment of the first line is broken again.
I tried using inspect.cleandoc and textwrap.dedent to alleviate this, but could not manage to fix the indentation issue. But perhaps this is the subject of another question.
Edit: After reading this article, I found a single line solution:
f_str_nl = lambda object: f"{chr(10) + str(object)}" # add \n directly
# f_str_nl = lambda object: f"{os.linesep + str(object)}" # add \r\n on windows
print(f"{f_str_nl(some_fairly_long_named_arr) = !s}")
which outputs:
f_str_nl(some_fairly_long_named_arr) =
[[0.26616956 0.59973262]
[0.86601261 0.10119292]
[0.94125617 0.9318651 ]
[0.10401072 0.66893025]]
The only caveat is that the name of the object gets prepended by the name of the custom lambda function, f_str_nl.
I also found that a similar question was already asked here.

Passing a well formulated criteria as a first class object in Python to an inner np.where() call?

How can I make this a dynamic or first class function that essentially passes the criteria to a np.where() call?
def num_assets(obj, criteria=None):
"""
A flexible wrapper to return the number of assets in a portfolio.
# list (asset names or characteristics)
>>> num_assets([1, 2, 3, 4, 5])
5
# number (pre-specification)
>>> num_assets(7)
7
# column vector (weights)
>>> num_assets(np.zeros(shape=(3,1)))
3
# matrix (covariance matrix)
>>> num_assets(np.eye(10))
10
# criteria.
>>> num_assets([1, 2, 3, 4, 5], '> 3')
??? I AM STUCK HERE AND NEED SOME HELP! Should return 2
"""
if criteria is None:
if myc.is_iterable(obj):
shape_obj = np.shape(obj)
return max(shape_obj)
elif myc.is_number(obj):
return myc.is_number(obj, True)
else:
return np.where(criteria)
myc.is_iterable() is essentially a boolean function containing a try except clause to iter notifying me if obj is iterable. myc.is_number() is telling me whether the obj is a number and when I pass the True parameter, it parses the number (in case obj is a string). I consider myself a newbie and know that this should not be too difficult a problem to solve, its just that I am not sure what advanced area of Python I need to apply to solve the criteria type problem (first class objects, meta programming, ...)? Also, if there is a cleaner more pythonic way of formulating the problem/getting the answer, contributions would be most welcome.
Thanks.

If I understand what you mean correctly, then I think this will do what you want.
if criteria is None:
what you already have
else:
obj = np.asarray(obj)
return np.sum(eval('obj'+criteria))
Its not elegant to have to use eval, but I think that's the easiest way to do this. For your example of criteria='>3', this becomes np.sum(obj>3), which is np.sum([False,False,False,True,True]), which is 2. where is not needed here.

How to return multiple strings from a script to the rule sequence in booggie 2?

This is an issue specific to the use of python scripts in booggie 2.
I want to return multiple strings to the sequence and store them there in variables.
The script should look like this:
def getConfiguration(config_id):
""" Signature: getConfiguration(int): string, string"""
return "string_1", "string_2"
In the sequence I wanna have this:
(param_1, param_2) = getConfiguration(1)
Please note: The booggie-project does not exist anymore but led to the development of Soley Studio which covers the same functionality.

Scripts in booggie 2 are restricted to a single return value.
But you can return an array which then contains your strings.
Sadly Python arrays are different from GrGen arrays so we need to convert them first.
So your example would look like this:
def getConfiguration(config_id):
""" Signature: getConfiguration(int): array<string>"""
#TypeHelper in booggie 2 contains conversion methods from Python to GrGen types
return TypeHelper.ToSeqArray(["string_1", "string_2"])

return a tuple
return ("string_1", "string_2")
See this example
In [124]: def f():
.....: return (1,2)
.....:
In [125]: a, b = f()
In [126]: a
Out[126]: 1
In [127]: b
Out[127]: 2

Still, it's not possible to return multiple values but a python list is now converted into a C#-array that works in the sequence.
The python script itself should look like this
def getConfiguration(config_id):
""" Signature: getConfiguration(int): array<string>"""
return ["feature_1", "feature_2"]
In the sequence, you can then use this list as if it was an array:
config_list:array<string> # initialize array of string
(config_list) = getConfigurationList(1) # assign script output to that array
{first_item = config_list[0]} # get the first string("feature_1")
{second_item = config_list[1]} # get the second string("feature_2")

For the example above I recommend using the following code to access the entries in the array (in the sequence):
config_list:array<string> # initialize array of string
(config_list) = getConfigurationList(1) # assign script output to that array
{first_item = config_list[0]} # get the first string("feature_1")
{second_item = config_list[1]} # get the second string("feature_2")