Develop Reference

using `fsolve` to solve m equations with n unknowns where n<m

Imagine I have two equations with one unknown and I want to use fsolve to solve it:
0 = 0.5*x[0]**2-2
0 = 2-x
Clearly the answer is x=2. I have tried this
import numpy as np; from scipy.optimize import fsolve
def f(x):
r = np.zeros(2)
r[0] = 0.5*x[0]**2-2
r[1] = 2-x[0]
return r
fsolve(f,[0.5])
The error message is "The array returned by a function changed size between calls"
I can't see what is going wrong here. How do I solve this problem?
In general, How do I solve equations where the number of variables is less than the number of equations.
Here is the full message
Traceback (most recent call last):
File "<ipython-input-37-e4f77791f3f6>", line 12, in <module>
fsolve(f,[0.5])
File "... anaconda3/lib/python3.7/site-packages/scipy/optimize/minpack.py", line 148, in fsolve
res = _root_hybr(func, x0, args, jac=fprime, **options)
File ".... /anaconda3/lib/python3.7/site-packages/scipy/optimize/minpack.py", line 227, in _root_hybr
ml, mu, epsfcn, factor, diag)
ValueError: The array returned by a function changed size between calls

In case of overdetermined system (the number of equations is greater than the number of variables)you need to use, for example, least squares approach. In such cases, there are usually no solution in traditional sense. And we need to define, what should we treat as a solution of the system.
Let you have a system of two equations with one scalar variable:
f(x) = 0
g(x) = 0
This system usually inconsistent and have no solution in traditional sense.
Lets add some values eps1 and eps2 to the right part of the system:
f(x) = 0 + eps1
g(x) = 0 + eps2
eps1 and eps2 are some values;
Now, lets find such x when eps1^2 + eps2^2 is rises its minimum value; that will be a solution of the system
in least squares sense.
To get such solution using scipy you can use least_square function.
Lets look at the following piece of code, that solves your system of equations:
import numpy as np
from scipy.optimize import fsolve, least_squares
def f(x):
r = np.zeros(2)
r[0] = 0.5*x**2-2
r[1] = 2-x
return r
least_squares(f, [0.0])
Result:
active_mask: array([0.])
cost: 5.175333019854869e-20
fun: array([ 2.87759150e-10, -1.43879575e-10])
grad: array([7.19397879e-10])
jac: array([[ 2.00000001],
[-1. ]])
message: '`gtol` termination condition is satisfied.'
nfev: 6
njev: 6
optimality: 7.193978788924559e-10
status: 1
success: True
x: array([2.])

optimization with python cvxopt

I am trying to minimize the portfolio variance using Python's cvxopt. However, after lots of trying, it doesn't seem to work. The function and my code and the error are pasted below. Thanks for helping!
the minimize problem
objective function: min x.dot(sigma_mv).dot(x.T)
the constraint condition is all x>=0, sum(X) = 1
sigma_mv is the covariance matrix of 800*800, dim = 800
code
dim = sigma_mv.shape[0]
P = 2*sigma_mv
q = np.matrix([0.0])
G = -1*np.identity(dim)
h = np.matrix(np.zeros((dim,1)))
sol = solvers.qp(P,q,G,h)
Traceback (most recent call last):
File "<ipython-input-47-a077fa141ad2>", line 6, in <module>
sol = solvers.qp(P,q)
File "D:\spyder\lib\site-packages\cvxopt\coneprog.py", line 4470, in qp
return coneqp(P, q, G, h, None, A, b, initvals, kktsolver = kktsolver, options = options)
File "D:\spyder\lib\site-packages\cvxopt\coneprog.py", line 1822, in coneqp
raise ValueError("use of function valued P, G, A requires a "\
ValueError: use of function valued P, G, A requires a user-provided kktsolver

You have both equality and inequality constraints so you need to provide all the arguments to the built-in qp solver
Gx <=h
Ax=b
Here x>=0 can be written as -x<=0 So, G matrix will look like -1*(Identity matrix)
and h will a 0 vector
Similarly, your A will be an Identity matrix and b will be a unity vector(all elements =1)
Finally, solve expression should look like :
sol=solvers.qp(P, q, G, h, A, b)

defining a fuction to be used in solving equation from a data file

I am completely new to python and in fact any fundamental programming language, I use Mathematica for my all my symbolic and numeric calculations. I am learning to work with python and finding it really awesome! Here is a problem I am trying to solve but stuck without a clue!
I have a data file for example
0. 1.
0.01 0.9998000066665778
0.02 0.9992001066609779
... ..
Which just the {t, Cos[2t]}.
I want to define a function out of this data and use it in solving an equation in python. My Mathematica intuition tells me that I should define the function like:
iFunc[x_] = Interpolation[iData, x]
and rest of the job is easy. for instance
NDSolve[{y''[x] + iFunc[x] y[x] == 0, y[0] == 1, y[1] == 0}, y, {x, 0, 1}]
Solves the equation easily. (I have not tried with more complicated cases though).
Now how to do the job in python and also accuracy is an important issue for me. So, now I would like to ask two questions.
1. Is this the most accurate method in Mathematica?
2. And what is the equivalent of more accurate way to do the problem in python?
Here is my humble attempt to solve the problem (with a lot of input from StackOverflow) where the definition with cos(2t) works:
from scipy.integrate import odeint
import numpy as np
import matplotlib.pyplot as plt
from math import cos
from scipy import interpolate
data = np.genfromtxt('cos2t.dat')
T = data[:,0] #first column
phi = data[:,1] #second column
f = interpolate.interp1d(T, phi)
tmin = 0.0# There should be a better way to define from the data
dt = 0.01
tmax = 2*np.pi
t = np.arange(tmin, tmax, dt)
phinew = f(t) # use interpolation function returned by `interp1d`
"""
def fun(z, t):
x, y = z
return np.array([y, -(cos(2*t))*x ])
"""
def fun(z, t):
x, y = z
return np.array([y, -(phinew(t))*x ])
sol1 = odeint(fun, [1, 0], t)[..., 0]
# for checking the plots
plt.plot(t, sol1, label='sol')
plt.show()
*When I run the code with interpolated function from cos(2t) data, is not working...the error message tell
Traceback (most recent call last): File "testde.py", line 30,
in <module> sol1 = odeint(fun, [1, 0], t)[..., 0]
File "/home/archimedes/anaconda3/lib/python3.6/site-packages/scip‌​y/integrate/odepack.‌​py",
line 215, in odeint ixpr, mxstep, mxhnil, mxordn, mxords)
File "testde.py",
line 28, in fun return np.array([y, -(phinew(t))*x ])
TypeError: 'numpy.ndarray' object is not callable.
I really can't decipher them. Please help...

In Mathematica, the usual way is simply
iFunc = Interpolation[iData]
Interpolation[iData] already returns a function.

To sub-question 2
With
t = np.arange(tmin, tmax, dt)
phinew = f(t) # use interpolation function returned by `interp1d`
equivalent to
phinew = np.array([ f(s) for s in t])
you construct phinew not as callable function but as array of values, closing the circle array to interpolation function to array. Use f which is a scalar function directly in the derivatives function,
def fun(z, t):
x, y = z
return np.array([y, -f(t)*x ])

Is it possible to compute double integral using scipy.integrate.fixed_quad?

I am trying to compute a double integral (over a triangle with nodes at (0,0), (0,1), (1,0)) using Gaussian quadrature of order n. However, running
import scipy.integrate as integrate
f = lambda x,y: x+y
inside = lambda x: integrate.fixed_quad(f, 0, 1-x, args=(x,), n=5)[0]
outside = integrate.fixed_quad(inside, 0, 1, n=5)
gives
Traceback (most recent call last):
File "", line 1, in
File "/Users/username/anaconda/lib/python3.5/site-packages/scipy/integrate/quadrature.py", line 82, in fixed_quad
return (b-a)/2.0 * np.sum(w*func(y, *args), axis=0), None
File "", line 1, in
File "/Users/username/anaconda/lib/python3.5/site-packages/scipy/integrate/quadrature.py", line 78, in fixed_quad
if np.isinf(a) or np.isinf(b):
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
This is the second part of the question Can scipy.integrate.fixed_quad compute integral with functional boundaries?.

The answer to your question is, yes, under certain conditions.
For demonstration purposes, I first choose different bounds than you (11 instead of 1 - x).
Generally, one can solve these types of integrals using dblquad:
area_dblquad = integrate.dblquad(lambda x, y: x + y, 0, 1, lambda x: 0, lambda x: 11)[0]
which here returns 66. That is not an option as you mentioned in the comments.
One can now do this integration stepwise and it works fine for quad as well as fixed_quad:
def integrand(x, y):
return x + y
def fint_quad(x):
return integrate.quad(integrand, 0, 11, args=(x, ))[0]
def fint_fixed_quad(x):
return integrate.fixed_quad(integrand, 0, 11, args=(x, ), n=5)[0]
res_quad = integrate.quad(lambda x: fint_quad(x), 0, 1)
res_fixed_quad = integrate.fixed_quad(lambda x: fint_fixed_quad(x), 0, 1, n=5)
They both return 66 as well, as expected. That shows that it can work to compute double integrals using scipy.integrate.fixed_quad.
However, when one now changes the upper bound back to the one you had (from 11 to 1 - x), it still works for quad but crashes for fixed_quad:
area_dblquad = integrate.dblquad(lambda x, y: x + y, 0, 1, lambda x: 0, lambda x: 1 - x)[0]
res_quad = integrate.quad(lambda x: fint_quad(x), 0, 1)
both return 0.333333..., the call with fixed_quad results in the error you received. One can understand the error by looking on the source code:
x, w = _cached_roots_legendre(n)
x = np.real(x)
if np.isinf(a) or np.isinf(b):
raise ValueError("Gaussian quadrature is only available for "
"finite limits.")
y = (b-a)*(x+1)/2.0 + a
return (b-a)/2.0 * np.sum(w*func(y, *args), axis=-1), None
When one prints a and b one gets:
a: 0
b: 1
a: 0
b: [ 0.95308992 0.76923466 0.5 0.23076534 0.04691008]
So for the call with 1-x, b is actually a numpy array with n elements and one cannot compare an array to infinity, that's why it crashes. Whether that is an intended behavior or a bug, I can't answer; might be worth opening an issue on github.

fixed_quad requires f to accept vector inputs. And the result should be the mapped values for the inputs (i.e. something like map(f, xs)). With this in mind, just ensure your inside function returns mapped values, and you're ready to go.
import scipy.integrate as integrate
f = lambda y,x: x+y
inside = lambda xs, n: np.array([integrate.fixed_quad(f, 0, 1-x, args=(x,), n=n)[0]
for x in np.array(xs)])
order = 5
outside = integrate.fixed_quad(inside, 0, 1, n=order, args=(order,))
Also, be careful with the order of arguments for your integrand. Judging from your code arg=(x,), it seems that you want the inner integral to be done along y dimension. The first argument of the integrand is the dimension along which it is integrated. So it should be lambda y,x instead (note that this is also the shape of integrand expected by dblquad).

root minuit2 contours with parameter limits

I'm trying to produce contour plots for parameters with physical limits using the Minuit2 minimizer which is a part of the ROOT data analysis framework. Unfortunately, Minuit2 seems intent on drifting the parameters into regions outside of their limits when I try to produce contour plots:
>>> from minuit2 import Minuit2
>>> def f(x,y):
... if x < 0 or y < 0:
... print 'x = %.2f, y = %.2f' % (x,y)
... raise Exception
... return x**2 + y**2
...
>>> m = Minuit2(f)
>>> m.limits['x'] = 0, 10
>>> m.limits['y'] = 0, 10
>>> m.migrad()
>>> xy = m.contour('x','y',3)
Info in <Minuit2>: MnMinos UP value has changed, need to update FunctionMinimum class
x = -9.95, y = 0.00
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in f
Exception
Has anybody else dealt with this or a similar problem? Are there any workarounds?
I've already asked this question on the ROOT forums, but I thought there might also be some stack overflow users who have dealt with this or a similar issue.

Try your example without raising an exception
def f(x,y):
return x ** 2 + y ** 2
and you will get reasonable xy contour points (i.e. within 1e-3 of the true contour).
Note that the parameter sigmas=3 in your contour call m.contour('x', 'y', 3) means that the contour for sigmas ** 2 == 9 will be computed and that contour points along the parameter limits are computed. As far as I can see this is not mentioned in the contour() pyminuit documentation).
In your example the contour starts at (0, 0), goes up to (3, 0), along the circle to (0, 3), and back to (0, 0).
A common method is to implement parameter limits (arbitrary shapes, not only min / max) in your cost function by returning very high values for excluded parameters:
def f(x,y):
if x < 0 or y < 0:
return 1e10
return x ** 2 + y ** 2
This does throw the optimizer out of the forbidden regions, but it does not prevent it to probe them sometimes (i.e. evaluate f there).
I don't know why contour() should strictly respect the limits you set via
m.limits['x'] = 0, 10
m.limits['y'] = 0, 10
Here's a short description of the contour algorithm used by Minuit (and Minuit2) and here is the documentation for the Minuit2 code in ROOT, I did not manage to find the actual C file showing the implementation.