This is most likely because I do not know how to use the standard scipy expect function method. When I use:
from scipy.stats import uniform
uniform.expect(lambda k: k**2,lb=-0.2,ub=0.2)
I got value : 0.0026666666666666666
If I use manual calculation:
np.mean(np.random.uniform(-0.2,0.2,1000)**2)
I got 0.013235491320680141, which is the right value I expect. So what did I do wrong with standard expect or integral function? Any help please.
If you look at the documentation for expect, ub and lb do not mean what you think they do. They are bounds on the integral, not parameters for the distribution.
You actually want:
scipy.stats.uniform(loc=-0.2, scale=0.4).expect(lambda x: x**2)
Related
I need to find the root of a multidimentional function F(x), I'm using the scipy function scipy.optimization.root(...,method=''), which allows me to select different methods for the solution. However, for some problems it becomes slow and not convergent, maybe it would be useful to try an alternative package. Do you know some of them?
Generally, the more you know about the problem the better. For example you may know an approximate range in which the root occurs. Then you may first run a brute search (using np.linspace for example) to find a good starting point for the method you want to use. Example:
Let's say you have a function like
def f(x):
return np.exp(-x)*(x-1)**4
scipy will fail to find a root if you start at x0=5, because of the exponential.
However, if you know that the solution is somewhere in (-10,10), you can do something like
X=np.linspace(-10,10,10)
x0 = X[ np.argmin( np.abs( f(X) ) ) ]
from scipy.optimize import root
y=root(f,x0)
print(y.x)
and you get a nice result (fast!), because np.argmin( np.abs( f(X) ) ) gives you the argument of X where f is closest to 0.
You have to keep in mind that such "tricks" are also dangerous if you use them without triple checking, and you always should have some intuition (or even better an analytical approximation) on what you expect.
I'm currently trying to solve numerically a minimization problem and I tried to use the optimization library available in SciPy.
My function and derivative are a bit too complicated to be presented here, but they are based on the following functions, the minimization of which do not work either:
def func(x):
return np.log(1 + np.abs(x))
def grad(x):
return np.sign(x) / (1.0 + np.abs(x))
When calling the fmin_bfgs function (and initializing the descent method to x=10), I get the following message:
Warning: Desired error not necessarily achieved due to precision loss.
Current function value: 2.397895
Iterations: 0
Function evaluations: 24
Gradient evaluations: 22
and the output is equal to 10 (i.e. initial point). I suppose that this error may be caused by two problems:
The objective function is not convex: however I checked with other non-convex functions and the method gave me the right result.
The objective function is "very flat" when far from the minimum because of the log.
Are my suppositions true? Or does the problem come from anything else?
Whatever the error can be, what can I do to correct this? In particular, is there any other available minimization method that I could use?
Thanks in advance.
abs(x) is always somewhat dangerous as it is non-differentiable. Most solvers expect problems to be smooth. Note that we can drop the log from your objective function and then drop the 1, so we are left with minimizing abs(x). Often this can be done better by the following.
Instead of min abs(x) use
min t
-t <= x <= t
Of course this requires a solver that can solve (linearly) constrained NLPs.
I need to calculate exp(x**2) where x = numpy.arange(30,90). This raises the warning:
RuntimeWarning: overflow encountered in exp
inf
I cannot safely ignore this warning, but neither SymPy nor mpmath is a solution and I need to perform array operations so a Numpy solution would be my dream.
Does anyone know how to handle this problem?
You could use a data type that has the necessary range, for example decimal.Decimal:
>>> import numpy as np
>>> from decimal import Decimal
>>> x = np.arange(Decimal(30), Decimal(90))
>>> y = np.exp(x ** 2)
>>> y[-1]
Decimal('1.113246031563799750400684712E+3440')
But what are you using these numbers for? Could you avoid the exponentiation and work with logarithms? More detail about your problem would be helpful.
I think you can use this method to solve this problem:
Normalized
I overcome the problem in this method. Before using this method, my classify accuracy is :86%. After using this method, my classify accuracy is :96%!!!
It's great!
first:
Min-Max scaling
second:
Z-score standardization
These are common methods to implement normalization.
I use the first method. And I alter it. The maximum number is divided by 10.
So the maximum number of the result is 10. Then exp(-10) will be not overflow!
I hope my answer will help you !(^_^)
When I try to differentiate a symbol with SymPy I get the following
In : x=Symbol('x')
In : diff(x,x)
Out: 1
When I differentiate the symbol respect to its conjugate the result is
In [55]: diff(x,x.conjugate())
Out[55]: 0
However, when I try to differentiate the conjugate of the symbol SymPy doesn't do it
In : diff(x.conjugate(),x)
Out: Derivative(conjugate(x), x)
This is still correct, but the result should be zero. How can I make SimPy perform the derivative of a conjugate?
I'm not sure about the mathematics if diff(conjugate(x), x) should be zero. The fact that diff(x,x.conjugate()) gives zero has nothing to do with mathematics (and might even be considered a SymPy bug). It gives zero simply because x does not contain conjugate(x) (symbolically), so it sees it as a constant with respect to it. This is probably wrong, since x is not a constant with respect to conjugate(x). The fact that SymPy lets you take derivatives with respect to defined functions is probably a bug, actually. It is supposed to allow things like diff(f(x)**2, f(x)), where f = Function('f') is an undefined function, but for defined functions, it is probably mathematically incorrect (or at least not what you expect).
See http://docs.sympy.org/latest/modules/core.html?highlight=derivative#sympy.core.function.Derivative, particularly the section on derivatives wrt non-Symbols. To paraphrase, taking derivatives with respect to a function is just a notational convenience and does not represent a mathematical chain rule. Rather, something like diff(x, conjugate(x)) should be thought of as something like diff(x.subs(conjugate(x), dummy), dummy).subs(dummy, conjugate(x)).
Regarding conjugate(x).diff(x), this gives an unevaluated derivative because no derivative is defined for conjugate. I'm not sure if any closed-form answer is possible here anyway. Probably this is the most useful thing that SymPy could return. I can't find any good answers anywhere as to what a reasonable answer for this should be (you should ask on math SE to get a better answer about it).
I face a problem in scipy 'leastsq' optimisation routine, if i execute the following program it says
raise errors[info][1], errors[info][0]
TypeError: Improper input parameters.
and sometimes index out of range for an array...
from scipy import *
import numpy
from scipy import optimize
from numpy import asarray
from math import *
def func(apar):
apar = numpy.asarray(apar)
x = apar[0]
y = apar[1]
eqn = abs(x-y)
return eqn
Init = numpy.asarray([20.0, 10.0])
x = optimize.leastsq(func, Init, full_output=0, col_deriv=0, factor=100, diag=None, warning=True)
print 'optimized parameters: ',x
print '******* The End ******'
I don't know what is the problem with my func optimize.leastsq() call, please help me
leastsq works with vectors so the residual function, func, needs to return a vector of length at least two. So if you replace return eqn with return [eqn, 0.], your example will work. Running it gives:
optimized parameters: (array([10., 10.]), 2)
which is one of the many correct answers for the minimum of the absolute difference.
If you want to minimize a scalar function, fmin is the way to go, optimize.fmin(func, Init).
The issue here is that these two functions, although they look the same for a scalars are aimed at different goals. leastsq finds the least squared error, generally from a set of idealized curves, and is just one way of doing a "best fit". On the other hand fmin finds the minimum value of a scalar function.
Obviously yours is a toy example, for which neither of these really makes sense, so which way you go will depend on what your final goal is.
Since you want to minimize a simple scalar function (func() returns a single value, not a list of values), scipy.optimize.leastsq() should be replaced by a call to one of the fmin functions (with the appropriate arguments):
x = optimize.fmin(func, Init)
correctly works!
In fact, leastsq() minimizes the sum of squares of a list of values. It does not appear to work on a (list containing a) single value, as in your example (even though it could, in theory).
Just looking at the least squares docs, it might be that your function func is defined incorrectly. You're assuming that you always receive an array of at least length 2, but the optimize function is insanely vague about the length of the array you will receive. You might try writing to screen whatever apar is, to see what you're actually getting.
If you're using something like ipython or the python shell, you ought to be getting stack traces that show you exactly which line the error is occurring on, so start there. If you can't figure it out from there, posting the stack trace would probably help us.