Related
Any help to compute this integration, F function is defined using the f function which involves the first integration, finally, integrate F.
from scipy.integrate import quad
f = lambda x,a : a**2*x
def F(s,a):
return quad(f,0,s,args=(a,))
quad(F,0,5,args=(4,))
Got the error:
2 def F(s,a):
3 return quad(f,0,s,args=(a,))
----> 4 quad(F,0,5,args=(4,))
5
446 if points is None:
447 if infbounds == 0:
--> 448 return _quadpack._qagse(func,a,b,args,full_output,epsabs,epsrel,limit)
449 else:
450 return _quadpack._qagie(func,bound,infbounds,args,full_output,epsabs,epsrel,limit)
TypeError: must be real number, not tuple
Have a look at the return values of scipy.integrate.quad:
Returns:
y: float The integral of func from a to b.
abserr: float An estimate of the absolute error in the result.
...
So there are multiple return values (a tuple) and that's why you're getting the TypeError: must be real number, not tuple message.
I guess, you're just interested in the integral value quad(...)[0] so that's what your F should return:
from scipy.integrate import quad
f = lambda x, a: a**2 * x
F = lambda x, a: quad(f, 0, x, args=(a,))[0]
I = quad(F, 0, 5, args=(4,))
print(I)
Which prints:
(333.33333333333337, 3.700743415417189e-12)
Another way of looking at the problem is to realize that you're integrating the function a**2 * y over the triangle spanned by the points [0, 0], [5, 0], and [5, 5]. You could then use triangle quadrature from quadpy (one of my projects) to compute the value:
import quadpy
a = 4.0
def g(x):
return a ** 2 * x[1]
scheme = quadpy.triangle.dunavant_05()
val = scheme.integrate(g, [[0.0, 0.0], [5.0, 0.0], [5.0, 5.0]])
print(val)
This should require a much fewer function evaluations that the nested quad approach.
I am solving the following optimization problem:
with this Python code:
from scipy.optimize import minimize
import math
def f(x):
return math.log(x[0]**2 + 1) + x[1]**4 + x[0]*x[2]
x0 = [0, 0, 0]
cons=({'type': 'ineq',
'fun': lambda x: x[0]**3 - x[1]**2 - 1},
{'type': 'ineq',
'fun': lambda x: x[0]},
{'type': 'ineq',
'fun': lambda x: x[2]})
res = minimize(f, x0, constraints=cons)
print res
I am getting an error
message: 'Inequality constraints incompatible'
What can cause this error?
The issue seems to be with your initial guess. If I change your starting values to
x0 = [1.0, 1.0, 1.0]
Then your code will execute fine (at least on my machine)
Python 3.5.1 (v3.5.1:37a07cee5969, Dec 6 2015, 01:54:25) [MSC v.1900 64 bit (AMD64)] on win32
message: 'Optimization terminated successfully.'
njev: 10
jac: array([ 1., 0., 1., 0.])
fun: 0.6931471805582502
nit: 10
status: 0
x: array([ 1.00000000e+00, -1.39724765e-06, 1.07686548e-14])
success: True
nfev: 51
Scipy's optimize module has lots of options. See the documentation or this tutorial. Since you didn't specify the method here, it will use Sequential Least SQuares Programming (SLSQP). Alternatively, you could use the Trust-Region Constrained Algorithm (trust-const).
For this problem, I found that trust-const seemed much more robust to starting values than SLSQP, handling starting values from [-2,-2,-2] to [10,10,10], although negative initial values resulted in increased iterations, as you'd expect. Negative values below -2 exceeded the max iterations, although I suspect might still converge if you increased max iterations, although specifying negative values at all for x1 and x3 is kind of silly, of course, I just did it to get a sense of how robust it was to a range of starting values.
The specifications for SLSQP and trust-const are conceptually the same, but the syntax is a little different (in particular, note the use of NonlinearConstraint).
from scipy.optimize import minimize, NonlinearConstraint, SR1
def f(x):
return math.log(x[0]**2 + 1) + x[1]**4 + x[0]*x[2]
constr_func = lambda x: np.array( [ x[0]**3 - x[1]**2 - 1,
x[0],
x[2] ] )
x0=[0.,0.,0.]
nonlin_con = NonlinearConstraint( constr_func, 0., np.inf )
res = minimize( f, x0, method='trust-constr',
jac='2-point', hess=SR1(),
constraints = nonlin_con )
Here are the results, edited for conciseness:
fun: 0.6931502233468916
message: '`gtol` termination condition is satisfied.'
x: array([1.00000063e+00, 8.21427026e-09, 2.40956900e-06])
Note that the function value and x values are the same as in #CoryKramer's answer. The x array may look superficially different at first glance, but both answers round to [1, 0, 0].
I am attempting to understand the behavior of the constraints in scipy.optimize.minimize:
First, I create 4 assets and 100 scenarios of returns. The average returning funds are in order best to worse D > B > A > C
#seed first
np.random.seed(1)
df_returns = pd.DataFrame(np.random.rand(100,4) - 0.25, columns =list('ABCD'))
df_returns.head()
A B C D
0 0.167022 0.470324 -0.249886 0.052333
1 -0.103244 -0.157661 -0.063740 0.095561
2 0.146767 0.288817 0.169195 0.435220
3 -0.045548 0.628117 -0.222612 0.420468
4 0.167305 0.308690 -0.109613 -0.051899
and a set of weights
weights = pd.Series([0.25, 0.25, 0.25, 0.25], index=list('ABCD'))
0
A 0.25
B 0.25
C 0.25
D 0.25
we create an objective function:
def returns_objective_function(weights, df_returns):
result = -1. * (df_returns * weights).mean().sum()
return result
and constraints and bounds
cons = ({'type': 'eq', 'fun': lambda weights: np.sum(weights) -1 })
bnds = ((0.01, .8), (0.01, .8), (0.01, .8), (0.01, .75))
Let's optimize
optimize.minimize(returns_objective_function, weights, (df_returns),
bounds=bnds, constraints=cons, method= 'SLSQP')
And we get success.
status: 0
success: True
njev: 8
nfev: 48
fun: -0.2885398923185326
x: array([ 0.01, 0.23, 0.01, 0.75])
message: 'Optimization terminated successfully.'
jac: array([-0.24384782, -0.2789166 , -0.21977262, -0.29300382, 0. ])
nit: 8
Now I wish to add constraints starting with a basic inequality:
scipy.optimize.minimize documentation states
Equality constraint means that the constraint function result is to be zero whereas inequality means that it is to be non-negative.
cons = (
{'type': 'eq', 'fun': lambda weights: np.sum(weights) -1 }
,{'type': 'ineq', 'fun': lambda weights: np.sum(weights) + x}
)
Depending on x, I get unexpected behavior.
x = -100
Based on the bounds, weights can be a maximum of 3.15 and, of course, must sum to 1 by the first equality constraint np.sum(weights) - 1, but, as a result, np.sum(weights) + x would always be negative. I believe no solution should be found, yet scipy.optimize.minimize returns success.
With a simpler model I get the same behavior:
x = [1,2]
optimize.minimize(
lambda x: x[0]**2+x[1]**2,
x,
constraints = (
{'type':'eq','fun': lambda x: x[0]+x[1]-1},
{'type':'ineq','fun': lambda x: x[0]-2}
),
bounds = ((0,None),(0,None)),
method='SLSQP')
with results:
nfev: 8
fun: 2.77777777777712
nit: 6
jac: array([ 3.33333334e+00, 2.98023224e-08, 0.00000000e+00])
x: array([ 1.66666667e+00, 1.39888101e-14])
success: True
message: 'Optimization terminated successfully.'
status: 0
njev: 2
There should be some flag that this is an infeasible solution.
SLSQP is also available from R:
> slsqp(c(1,2),
+ function(x) {x[1]^2+x[2]^2},
+ heq=function(x){x[1]+x[2]-1},
+ hin=function(x){x[1]-2},
+ lower=c(0,0))
$par
[1] 1.666667e+00 4.773719e-11
$value
[1] 2.777778
$iter
[1] 105
$convergence
[1] -4
$message
[1] "NLOPT_ROUNDOFF_LIMITED: Roundoff errors led to a breakdown of the optimization algorithm. In this case, the returned minimum may still be useful. (e.g. this error occurs in NEWUOA if one tries to achieve a tolerance too close to machine precision.)"
At least we see some warning signals here.
I have the following optimization scheme implemented under NNLS
in scipy.
import numpy as np
from scipy.optimize import nnls
from scipy import stats
#Define problem
A = np.array([[60., 90., 120.],
[30., 120., 90.]])
b = np.array([6700.5, 699.,])
# Add ones to ensure the solution sums to 1
b = np.hstack([b,1.0])
A = np.vstack([A,np.ones(3)])
x, rnorm = nnls(A,b)
print x
# the solution is
# [ 93.97933792 0. 0. ]
# we expect it to sum to 1 if it's not skewed
As you can see the b vector is much higher than values in A.
My question is what's the best/reasonable way to scale A and b so that the solution
is not skewed.
Note that both A and b are gene expression raw data without pre-processing.
If you want to include the equality constraint, you can't really use the nnls routine, since it doesn't cater for equalities. If you are limited to what's on offer in scipy, you can use this:
import numpy as np
from scipy.optimize import minimize
#Define problem
A = np.array([[60., 90., 120.],
[30., 120., 90.]])
b = np.array([6700.5, 699.,])
#-----------------------------
# I tried rescaling the data by adding this two lines,
# so that they're in same scale.
# but why the solution is different?
# x: array([ 1., 0., 0.])
# What's the correct way to go?
#-----------------------------
# A = A/np.linalg.norm(A,axis=0)
# b = b/np.linalg.norm(b)
def f(x):
return np.linalg.norm(A.dot(x) - b)
cons ={'type': 'eq',
'fun': lambda x: sum(x) - 1}
x0 = [1, 0, 0] # initial guess
minimize(f, x0, method='SLSQP', bounds=((0, np.inf),)*3, constraints=cons)
Output:
status: 0
success: True
njev: 2
nfev: 10
fun: 6608.620222860367
x: array([ 0., 0., 1.])
message: 'Optimization terminated successfully.'
jac: array([ -62.50927734, -100.675354 , -127.78314209, 0. ])
nit: 2
This minimises the objective function directly while also imposing the equality constraint you're interested in.
If speed is important, you can add the jacobian and hessian information, or even better, use a proper QP solver, as supplied by cvxopt.
I'm trying to minimize function, that returns a vector of values,
and here is an error:
setting an array element with a sequence
Code:
P = np.matrix([[0.3, 0.1, 0.2], [0.01, 0.4, 0.2], [0.0001, 0.3, 0.5]])
Ps = np.array([10,14,5])
def objective(x):
x = np.array([x])
res = np.square(Ps - np.dot(x, P))
return res
def main():
x = np.array([10, 11, 15])
print minimize(objective, x, method='Nelder-Mead')
At these values of P, Ps, x function returns [[ 47.45143225 16.81 44.89 ]]
Thank you for any advice
UPD (full traceback)
Traceback (most recent call last):
File "<ipython-input-125-9649a65940b0>", line 1, in <module>
runfile('C:/Users/Roark/Documents/Python Scripts/optimize.py', wdir='C:/Users/Roark/Documents/Python Scripts')
File "C:\Anaconda\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 585, in runfile
execfile(filename, namespace)
File "C:/Users/Roark/Documents/Python Scripts/optimize.py", line 28, in <module>
main()
File "C:/Users/Roark/Documents/Python Scripts/optimize.py", line 24, in main
print minimize(objective, x, method='Nelder-Mead')
File "C:\Anaconda\lib\site-packages\scipy\optimize\_minimize.py", line 413, in minimize
return _minimize_neldermead(fun, x0, args, callback, **options)
File "C:\Anaconda\lib\site-packages\scipy\optimize\optimize.py", line 438, in _minimize_neldermead
fsim[0] = func(x0)
ValueError: setting an array element with a sequence.
UPD2: function should be minimized (Ps is a vector)
If you want you resulting vector to be a vector containing only 0s, you can use fsolve to do so. To do that will require modifying your objective function a little bit to get the input and output into the same shape:
import scipy.optimize as so
P = np.matrix([[0.3, 0.1, 0.2], [0.01, 0.4, 0.2], [0.0001, 0.3, 0.5]])
Ps = np.array([10,14,5])
def objective(x):
x = np.array([x])
res = np.square(Ps - np.dot(x, P))
return np.array(res).ravel()
Root = so.fsolve(objective, x0=np.array([10, 11, 15]))
objective(Root)
#[ 5.04870979e-29 1.13595970e-28 1.26217745e-29]
Result: The solution is np.array([ 31.95419775, 41.56815698, -19.40894189])
Your objective function needs to return a scalar value, not a vector. You probably want to return the sum of squared errors rather than the vector of squared errors:
def objective(x):
res = ((Ps - np.dot(x, P)) ** 2).sum()
return res
Use least_squares. This will require to modify the objective a bit to return differences instead of squared differences:
import numpy as np
from scipy.optimize import least_squares
P = np.matrix([[0.3, 0.1, 0.2], [0.01, 0.4, 0.2], [0.0001, 0.3, 0.5]])
Ps = np.array([10,14,5])
def objective(x):
x = np.array([x])
res = Ps - np.dot(x, P)
return np.asarray(res).flatten()
def main():
x = np.array([10, 11, 15])
print(least_squares(objective, x))
Result:
active_mask: array([0., 0., 0.])
cost: 5.458917464129402e-28
fun: array([1.59872116e-14, 2.84217094e-14, 5.32907052e-15])
grad: array([-8.70414856e-15, -1.25943700e-14, -1.11926469e-14])
jac: array([[-3.00000002e-01, -1.00000007e-02, -1.00003682e-04],
[-1.00000001e-01, -3.99999999e-01, -3.00000001e-01],
[-1.99999998e-01, -1.99999999e-01, -5.00000000e-01]])
message: '`gtol` termination condition is satisfied.'
nfev: 4
njev: 4
optimality: 1.2594369966691647e-14
status: 1
success: True
x: array([ 31.95419775, 41.56815698, -19.40894189])