Why doesn't the following function take the inner h value that is defined in the function body and gives weird results (arbitrary h value)?
def diff(f): # def not define
h = 0.001
return (lambda x: (f(x+h) - f(x)) / h)
def sin_by_million (x):
return math.sin( 10 ** 6 *x)
>>> diff(sin_by_million) (0)
826.8795405320026
Instead of 1000000?
As per #ThierryLathuille comment, your step h is too big. In real life, you should adapt it based on the function and value at which you want the derivative.
Check out jax instead:
import jax
import jax.numpy as np
def sin_by_million(x):
return np.sin(1e6 * x)
Then:
>>> g = jax.grad(sin_by_million)
... g(0.0)
DeviceArray(1000000., dtype=float32)
The beauty of jax is that it actually compiles your call tree using chain rule, and produces some code (the calls after the first one are much, much faster). It also works on multivariate functions and complex code (with some rules though). And it works wonderfully well & fast on GPUs.
Related
I'm working in somewhat of a limited development environment. I'm writing a neural network in Python. I don't have access to numpy and as it is I can't even import the math module. So my options are limited. I need to calculate the sigmoid function, however I'm not sure how the exp() function works under the hood. I understand exponents and that I can use code like:
base = .57
exp = base ** exponent
However I'm not sure what exponent should be? How do functions like numpy.exp() calculate the exponent? This is what I need to replicate.
The exponential function exp(a) is equivalent to e ** a, where e is Euler's number.
>>> e = 2.718281828459045
>>> def exp(a):
... return e ** a
...
>>> import math # accuracy test
>>> [math.exp(i) - exp(i) for i in range(1, 12, 3)]
[0.0, 7.105427357601002e-15, 2.2737367544323206e-13, 1.4551915228366852e-11]
def sigmoid(z):
e = 2.718281828459
return 1.0/(1.0 + e**(-1.0*z))
# This is the formula for sigmoid in pure python
# where z = hypothesis. You have to find the value of hypothesis
you can use ** just fine for your use case it will work with both float and integer input
print(2**3)
8
print(2**0.5 )
1.4142135623730951
if you really need a drop in replacement for numpy.exp()
you can just make a function that behaves like it is written in the docs https://numpy.org/doc/stable/reference/generated/numpy.exp.html
from typing import List
def not_numpy_exp(x:[List[float],float]):
e = 2.718281828459045 # close enough
if type(x) == list:
return [e ** _x for _x in x]
else:
return e**x
how the exp() function works under the hood
If you mean math.exp from built-in module math in this place it does simply
exp(x, /)
Return e raised to the power of x.
where e should be understand as math.e (2.718281828459045). If import math is not allowed you might do
pow(2.718281828459045, x)
instead of exp(x)
I was looking for a Python library function which computes multinomial coefficients.
I could not find any such function in any of the standard libraries.
For binomial coefficients (of which multinomial coefficients are a generalization) there is scipy.special.binom and also scipy.misc.comb. Also, numpy.random.multinomial draws samples from a multinomial distribution, and sympy.ntheory.multinomial.multinomial_coefficients returns a dictionary related to multinomial coefficients.
However, I could not find a multinomial coefficients function proper, which given a,b,...,z returns (a+b+...+z)!/(a! b! ... z!). Did I miss it? Is there a good reason there is none available?
I would be happy to contribute an efficient implementation to SciPy say. (I would have to figure out how to contribute, as I have never done this).
For background, they do come up when expanding (a+b+...+z)^n. Also, they count the ways of depositing a+b+...+z distinct objects into distinct bins such that the first bin contains a objects, etc. I need them occasionally for a Project Euler problem.
BTW, other languages do offer this function: Mathematica, MATLAB, Maple.
To partially answer my own question, here is my simple and fairly efficient implementation of the multinomial function:
def multinomial(lst):
res, i = 1, 1
for a in lst:
for j in range(1,a+1):
res *= i
res //= j
i += 1
return res
It seems from the comments so far that no efficient implementation of the function exists in any of the standard libraries.
Update (January 2020). As Don Hatch has pointed out in the comments, this can be further improved by looking for the largest argument (especially for the case that it dominates all others):
def multinomial(lst):
res, i = 1, sum(lst)
i0 = lst.index(max(lst))
for a in lst[:i0] + lst[i0+1:]:
for j in range(1,a+1):
res *= i
res //= j
i -= 1
return res
No, there is not a built-in multinomial library or function in Python.
Anyway this time math could help you. In fact a simple method for calculating the multinomial
keeping an eye on the performance is to rewrite it by using the characterization of the multinomial coefficient as a product of binomial coefficients:
where of course
Thanks to scipy.special.binom and the magic of recursion you can solve the problem like this:
from scipy.special import binom
def multinomial(params):
if len(params) == 1:
return 1
return binom(sum(params), params[-1]) * multinomial(params[:-1])
where params = [n1, n2, ..., nk].
Note: Splitting the multinomial as a product of binomial is also good to prevent overflow in general.
You wrote "sympy.ntheory.multinomial.multinomial_coefficients returns a dictionary related to multinomial coefficients", but it is not clear from that comment if you know how to extract the specific coefficients from that dictionary. Using the notation from the wikipedia link, the SymPy function gives you all the multinomial coefficients for the given m and n. If you only want a specific coefficient, just pull it out of the dictionary:
In [39]: from sympy import ntheory
In [40]: def sympy_multinomial(params):
...: m = len(params)
...: n = sum(params)
...: return ntheory.multinomial_coefficients(m, n)[tuple(params)]
...:
In [41]: sympy_multinomial([1, 2, 3])
Out[41]: 60
In [42]: sympy_multinomial([10, 20, 30])
Out[42]: 3553261127084984957001360
Busy Beaver gave an answer written in terms of scipy.special.binom. A potential problem with that implementation is that binom(n, k) returns a floating point value. If the coefficient is large enough, it will not be exact, so it would probably not help you with a Project Euler problem. Instead of binom, you can use scipy.special.comb, with the argument exact=True. This is Busy Beaver's function, modified to use comb:
In [46]: from scipy.special import comb
In [47]: def scipy_multinomial(params):
...: if len(params) == 1:
...: return 1
...: coeff = (comb(sum(params), params[-1], exact=True) *
...: scipy_multinomial(params[:-1]))
...: return coeff
...:
In [48]: scipy_multinomial([1, 2, 3])
Out[48]: 60
In [49]: scipy_multinomial([10, 20, 30])
Out[49]: 3553261127084984957001360
Here are two approaches, one using factorials, one using Stirling's approximation.
Using factorials
You can define a function to return multinomial coefficients in a single line using vectorised code (instead of for-loops) as follows:
from scipy.special import factorial
def multinomial_coeff(c):
return factorial(c.sum()) / factorial(c).prod()
(Where c is an np.ndarray containing the number of counts for each different object). Usage example:
>>> import numpy as np
>>> coeffs = np.array([2, 3, 4])
>>> multinomial_coeff(coeffs)
1260.0
In some cases this might be slower because you will be computing certain factorial expressions multiple times, in other cases this might be faster because I believe that numpy naturally parallelises vectorised code. Also this reduces the required number of lines in your program and is arguably more readable. If someone has the time to run speed tests on these different options then I'd be interested to see the results.
Using Stirling's approximation
In fact the logarithm of the multinomial coefficient is much faster to compute (based on Stirling's approximation) and allows computation of much larger coefficients:
from scipy.special import gammaln
def log_multinomial_coeff(c):
return gammaln(c.sum()+1) - gammaln(c+1).sum()
Usage example:
>>> import numpy as np
>>> coeffs = np.array([2, 3, 4])
>>> np.exp(log_multinomial_coeff(coeffs))
1259.999999999999
Your own answer (the accepted one) is quite good, and is especially simple. However, it does have one significant inefficiency: your outer loop for a in lst is executed one more time than is necessary. In the first pass through that loop, the values of i and j are always identical, so the multiplications and divisions do nothing. In your example multinomial([123, 134, 145]), there are 123 unneeded multiplications and divisions, adding time to the code.
I suggest finding the maximum value in the parameters and removing it, so those unneeded operations are not done. That adds complexity to the code but reduces the execution time, especially for short lists of large numbers. My code below executes multcoeff(123, 134, 145) in 111 microseconds, while your code takes 141 microseconds. That is not a large increase, but that could matter. So here is my code. This also takes individual values as parameters rather than a list, so that is another difference from your code.
def multcoeff(*args):
"""Return the multinomial coefficient
(n1 + n2 + ...)! / n1! / n2! / ..."""
if not args: # no parameters
return 1
# Find and store the index of the largest parameter so we can skip
# it (for efficiency)
skipndx = args.index(max(args))
newargs = args[:skipndx] + args[skipndx + 1:]
result = 1
num = args[skipndx] + 1 # a factor in the numerator
for n in newargs:
for den in range(1, n + 1): # a factor in the denominator
result = result * num // den
num += 1
return result
Starting Python 3.8,
since the standard library now includes the math.comb function (binomial coefficient)
and since the multinomial coefficient can be computed as a product of binomial coefficients
we can implement it without external libraries:
import math
def multinomial(*params):
return math.prod(math.comb(sum(params[:i]), x) for i, x in enumerate(params, 1))
multinomial(10, 20, 30) # 3553261127084984957001360
Lets say i define my function G,
def G(k, P, W):
return k**2*P*W**2
Where P and W are two functions that have independent variables of k and k is a defined number.
I am trying to integrate this from 0 to infinity
I = scipy.integrate.quad(G, 0, np.Inf)
inputting this into my console gives me the error,
G() takes exactly 3 arguments (2 given)
I tried using the arg() command, but it does not seem to change it and code remains stubborn. What am i doing wrong and what am i missing?
If I understand correctly, k is a constant. Then you can write:
k = 10
I = integrate.dblquad(lambda p,w: G(k,p,w), 0, np.Inf, lambda x: 0, lambda x: np.Inf)
Found it in the scipy documentation.
Besides, your integral looks divergent.
For symbolic integrals see sympy.integrate. It is a different library.
import * from sympy
k,P,W = symbols('k P W')
integrate(G(k,P,W),P,W)
Just started learning python, and was asked to define a python function that integrate a math function.
We were instructed that the python function must be in the following form: (for example, to calculate the area of y = 2x + 3 between x=1 and x=2 )
integrate( 2 * x + 3, 1, 2 )
(it should return the area below)
and we are not allowed to use/import any libraries other than math (and the built in integration tool is not allowed either).
Any idea how I should go about it? When I wrote the program, I always get x is undefined, but if I define x as a value ( lets say 0 ) then the 2*x+3 part in the parameters is always taken as a value instead of a math equation, so I can't really use it inside?
It would be very helpful, not just to this assignment, but many in the future if I know how a python function can take a math equation as parameter, so thanks alot.
Let's say your integration function looks like this:
def integrate(func, lo_x, hi_x):
#... Stuff to perform the integral, which will need to evaluate
# the passed function for various values of x, like this
y = func(x)
#... more stuff
return value
Then you can call it like this:
value = integrate(lambda x: 2 * x + 3, 1, 2)
edit
However, if the call to the integration function has to look exactly like
integrate( 2 * x + 3, 1, 2 )
then things are a bit trickier. If you know that the function is only going to be called with a polynomial function you could do it by making x an instance of a polynomial class, as suggested by M. Arthur Vaïsse in his answer.
Or, if the integrate( 2 * x + 3, 1, 2 ) comes from a string, eg from a command line argument or a raw_input() call, then you could extract the 2 * x + 3 (or whatever) from the string using standard Python string methods and then build a lambda function from that using exec.
Here come an implementation that fill the needs I think. It allow you to define mathematical function such as 2x+3 and propose an implementation of integral calculation by step as described here [http://en.wikipedia.org/wiki/Darboux_integral]
import math
class PolynomialEquation():
""" Allow to create function that are polynomial """
def __init__(self,coef):
"""
coef : coeficients of the polynome.
An equation initialized with [1,2,3] as parameters is equivalent to:
y = 1 + 2X + 3X²
"""
self.coef = coef
def __call__(self, x):
"""
Make the object callable like a function.
Return the value of the equation for x
"""
return sum( [self.coef[i]*(x**i) for i in range(len(self.coef)) ])
def step_integration(function, start, end, steps=100):
"""
Proceed to a step integration of the function.
The more steps there are, the more the approximation is good.
"""
step_size = (end-start)/steps
values = [start + i*step_size for i in range(1,steps+1)]
return sum([math.fabs(function(value)*step_size) for value in values])
if __name__ == "__main__":
#check that PolynomialEquation.value works properly. Assert make the program crash if the test is False.
#y = 2x+3 -> y = 3+2x -> PolynomialEquation([3,2])
eq = PolynomialEquation([3,2])
assert eq(0) == 3
assert eq(1) == 5
assert eq(2) == 7
#y = 1 + 2X + 3X² -> PolynomialEquation([1,2,3])
eq2 = PolynomialEquation([1,2,3])
assert eq2(0) == 1
assert eq2(1) == 6
assert eq2(2) == 17
print(step_integration(eq, 0, 10))
print(step_integration(math.sin, 0, 10))
EDIT : in truth the implementation is only the upper Darboux integral. The true Darboux integral could be computed if really needed by computing the lower Darboux integral ( replace range(1, steps+1) by range(steps) in step_integration function give you the lower Darboux function. And then increase the step parameter while the difference between the two Darboux function is greater than a small value depending on your precision need (could be 0.001 for example). Thus a 100 step integration is suppose to give you a decent approximation of the integral value.
I'm trying to solve the equation f(x) = x-sin(x) -n*t -m0
In this equation, n and m0 are attributes, defined in my class. Further, t is a constant integer in the equation, but it has to change each time.
I've solved the equation so i get a 'new equation'. I've imported scipy.optimize
def f(x, self):
return (x - math.sin(x) -self.M0 - self.n*t)
def test(self,t):
return fsolve(self.f, 1, args=(t))
Any corrections and suggestions to make it work?
I can see at least two problems: you've mixed up the order of arguments to f, and you're not giving f access to t. Something like this should work:
import math
from scipy.optimize import fsolve
class Fred(object):
M0 = 5.0
n = 5
def f(self, x, t):
return (x - math.sin(x) -self.M0 - self.n*t)
def test(self, t):
return fsolve(self.f, 1, args=(t))
[note that I was lazy and made M0 and n class members]
which gives:
>>> fred = Fred()
>>> fred.test(10)
array([ 54.25204733])
>>> import numpy
>>> [fred.f(x, 10) for x in numpy.linspace(54, 55, 10)]
[-0.44121095114838482, -0.24158955381855662, -0.049951288133726734,
0.13271070588400136, 0.30551399241764443, 0.46769772292130796,
0.61863201965219616, 0.75782574394219182, 0.88493255340251409,
0.99975517335862207]
You need to define f() like so:
def f(self, x, t):
return (x - math.sin(x) - self.M0 - self.n * t)
In other words:
self comes first (it always does);
then comes the current value of x;
then come the arguments you supply to fsolve().
You're using a root finding algorithm of some kind. There are several in common use, so it'd be helpful to know which one.
You need to know three things:
The algorithm you're using
The equation you're finding the roots for
The initial guess and range over which you're looking
You need to know that some combinations may not have any roots.
Visualizing the functions of interest can be helpful. You have two: a linear function and a sinusiod. If you were to plot the two, which sets of constants would give you intersections? The intersection is the root you're looking for.