Develop Reference

numpy, pass individual arguments to np.vectorized function

if i have an array of x values and want to multiply each x value with a different coefficients and sum the. although I want this operation to happen by passing a function that handles the summation and weighting. for example if i have x, coeffs and a function, custom_weight(x, a, b, c)
x = numpy.array([1, 2, 3, 4, 5, 6])
coeffs = numpy.array([[0.1, 0.2, 3.2], [4.5, 4.0, 0.005]]
def custom_weight(x, a, b, c):
return a*x**2 + (x+b)**3 + x*c
I want x to be broadcast for each inner array of coeffs. in this case the final result
should be an array with the shape (6, 2). for the first iteration of the custom_weight function
should look like this custom_weight(x[0], *(coeffs[0])) == custom_weight(1, 0.1, 0.2, 3.2). the same happens for all the other x's 2-6. then this happens again with the x's but now using the second set of coefficients.
I do realize that I could do this manually or numpy.vectorize in a certain way... but I specifically want to use a function in that form. what I want is some function that would look like so:
numpy.the_function(x, coeffs, axis=0, custom_weight)
# the_function should take each x value and pass it to custom_weight as the first arg.
# then pass the column of coeffs (because axis=0)
# to custom_weight but it should do this by unpacking the column into the args a, b, and c

The problem is more because your custome_weight function is not designed to be vectorized. You are looking for something like this:
def custom_weight(x, coeffs):
return coeffs # x**np.array([[2,3,1]]).T
Output:
array([[ 3.5 , 8.4 , 15.9 , 27.2 , 43.5 , 66. ],
[ 8.505, 50.01 , 148.515, 328.02 , 612.525, 1026.03 ]])

So after messing around, one solution I found was by vectorizing. transposing the coefficients when passing the arguments to custom_weight and then unpacking the coefficients and it will broadcasting and np.vectorize takes care of the rest.
import numpy as np
def custom_weight(x, a, b):
return a*x**2 + b
x = np.linspace(-1, 1, 100)
coeffs = np.array([[0.2, 0.6],
[1.2, 0.1]])
vec_custom_weight = np.vectorize(custom_weight)
results = vec_custom_weight(xs[:, np.newaxis], *coeffs.T).T

How to use `scipy.integrate.quad` to compute integral of a function which depends on the integral of another function

Any help to compute this integration, F function is defined using the f function which involves the first integration, finally, integrate F.
from scipy.integrate import quad
f = lambda x,a : a**2*x
def F(s,a):
return quad(f,0,s,args=(a,))
quad(F,0,5,args=(4,))
Got the error:
2 def F(s,a):
3 return quad(f,0,s,args=(a,))
----> 4 quad(F,0,5,args=(4,))
5
446 if points is None:
447 if infbounds == 0:
--> 448 return _quadpack._qagse(func,a,b,args,full_output,epsabs,epsrel,limit)
449 else:
450 return _quadpack._qagie(func,bound,infbounds,args,full_output,epsabs,epsrel,limit)
TypeError: must be real number, not tuple

Have a look at the return values of scipy.integrate.quad:
Returns:
y: float The integral of func from a to b.
abserr: float An estimate of the absolute error in the result.
...
So there are multiple return values (a tuple) and that's why you're getting the TypeError: must be real number, not tuple message.
I guess, you're just interested in the integral value quad(...)[0] so that's what your F should return:
from scipy.integrate import quad
f = lambda x, a: a**2 * x
F = lambda x, a: quad(f, 0, x, args=(a,))[0]
I = quad(F, 0, 5, args=(4,))
print(I)
Which prints:
(333.33333333333337, 3.700743415417189e-12)

Another way of looking at the problem is to realize that you're integrating the function a**2 * y over the triangle spanned by the points [0, 0], [5, 0], and [5, 5]. You could then use triangle quadrature from quadpy (one of my projects) to compute the value:
import quadpy
a = 4.0
def g(x):
return a ** 2 * x[1]
scheme = quadpy.triangle.dunavant_05()
val = scheme.integrate(g, [[0.0, 0.0], [5.0, 0.0], [5.0, 5.0]])
print(val)
This should require a much fewer function evaluations that the nested quad approach.

Python Sympy dot with the conjugate not same as norm squared

Why does sympy give false on the second Boolean? It correctly gives true on the first Boolean. I thought this last line would be the definition of the norm.
from sympy import *
eta_1, eta_2, m = 1, 1, 3
theta_1, theta_2 = symbols("theta_1 theta_2", real=True)
sigma_x = Matrix([[0, 1], [1, 0]])
sigma_y = Matrix([[0, -I], [I, 0]])
sigma_z = Matrix([[1, 0], [0, -1]])
H = eta_1*sin(theta_1)*sigma_x + eta_2*sin(theta_2)*sigma_y + (m-eta_1*cos(theta_1)-eta_2*cos(theta_2))*sigma_z
v = H.eigenvects()
l = v[0][0]
v = v[0][2][0]
n_normal = v/v.norm()
print(simplify(n_normal.norm()**2) == 1)
print(simplify(n_normal.dot(n_normal.H)))
print(simplify(n_normal.dot(n_normal.H)) == 1)
I think this has to do with the fact that sympy fails in that
simplify(abs(x**2)-x*conjugate(x))==0
gives false. Is there some other way to go around this problem, another way to define an inner product that does behave correctly. I'm doing some complicated physics calculations for my thesis and I would really like to check my results with sympy.
PS. I'm using sympy version 1.4dev.
Edit: I think the problem is with the fact that simplify doesn't realize that
$2*\cos(\theta_1)*cos(\theta_2) - 6*\cos(\theta_1) - 6*\cos(\theta_2) + 11>0\:.$
If I replace this in $n_normal$ with its absolute value it works. I think it is weird that the norm function does this correctly and the simplify of what essentially should be the norm doesn't.

Shortest distance between a point and a line in 3 d space

I am trying to find the minimum distance from a point (x0,y0,z0) to a line joined by (x1,y1,z1) and (x2,y2,z2) using numpy or anything in python. Unfortunately, all i can find on the net is related to 2d spaces and i am fairly new to python. Any help will be appreciated. Thanks in advance!

StackOverflow doesn't support Latex, so I'm going to gloss over some of the math. One solution comes from the idea that if your line spans the points p and q, then every point on that line can be represented as t*(p-q)+q for some real-valued t. You then want to minimize the distance between your given point r and any point on that line, and distance is conveniently a function of the single variable t, so standard calculus tricks work fine. Consider the following example, which calculates the minimum distance between r and the line spanned by p and q. By hand, we know the answer should be 1.
import numpy as np
p = np.array([0, 0, 0])
q = np.array([0, 0, 1])
r = np.array([0, 1, 1])
def t(p, q, r):
x = p-q
return np.dot(r-q, x)/np.dot(x, x)
def d(p, q, r):
return np.linalg.norm(t(p, q, r)*(p-q)+q-r)
print(d(p, q, r))
# Prints 1.0
This works fine in any number of dimensions, including 2, 3, and a billion. The only real constraint is that p and q have to be distinct points so that there is a unique line between them.
I broke the code down in the above example in order to show the two distinct steps arising from the way I thought about it mathematically (finding t and then computing the distance). That isn't necessarily the most efficient approach, and it certainly isn't if you want to know the minimum distance for a wide variety of points and the same line -- doubly so if the number of dimensions is small. For a more efficient approach, consider the following:
import numpy as np
p = np.array([0, 0, 0]) # p and q can have shape (n,) for any
q = np.array([0, 0, 1]) # n>0, and rs can have shape (m,n)
rs = np.array([ # for any m,n>0.
[0, 1, 1],
[1, 0, 1],
[1, 1, 1],
[0, 2, 1],
])
def d(p, q, rs):
x = p-q
return np.linalg.norm(
np.outer(np.dot(rs-q, x)/np.dot(x, x), x)+q-rs,
axis=1)
print(d(p, q, rs))
# Prints array([1. , 1. , 1.41421356, 2. ])
There may well be some simplifications I'm missing or other things that could speed that up, but it should be a good start at least.

This duplicates #Hans Musgrave solution, but imagine we know nothing of
'standard calculus tricks' that 'work fine' and also very bad at linear algebra.
All we know is:
how to calculate a distance between two points
a point on line can be represented as a function of two points and a paramater
we know find a function minimum
(lists are not friends with code blocks)
def distance(a, b):
"""Calculate a distance between two points."""
return np.linalg.norm(a-b)
def line_to_point_distance(p, q, r):
"""Calculate a distance between point r and a line crossing p and q."""
def foo(t: float):
# x is point on line, depends on t
x = t * (p-q) + q
# we return a distance, which also depends on t
return distance(x, r)
# which t minimizes distance?
t0 = sci.optimize.minimize(foo, 0.1).x[0]
return foo(t0)
# in this example the distance is 5
p = np.array([0, 0, 0])
q = np.array([2, 0, 0])
r = np.array([1, 5, 0])
assert abs(line_to_point_distance(p, q, r) - 5) < 0.00001
You should not use this method for real calculations, because it uses
approximations wher eyou have a closed form solution, but maybe it helpful
to reveal some logic begind the neighbouring answer.

python recursive vectorization with timeseries

I have a Timeseries (s) which need to be processed recursively to get a timeseries result (res). Here is my sample code:
res=s.copy()*0
res[1]=k # k is a constant
for i in range(2,len(s)):
res[i]=c1*(s[i]+s[i-1])/2 +c2*res[i-1]+c3*res[i-2]
where c1,c2,c3 are constants. It works properly but I'd like to use vectorization and I tried with:
res[2:]=c1*(s[2:]+s[1:-1])/2+c2*res[1:-1]+c3*res[0:-2]
but I get "ValueError: operands could not be broadcast together with shapes (1016) (1018) "
if I try with
res=c1*(s[2:]+s[1:-1])/2+c2*res[1:-1]+c3*res[0:-2]
doesn't give any error, but I don't get a correct result, because res[0] and res[1] have to be initialized before the calculation will take place.
Is there a way to process it with vectorization?
Any help will be appreciated, thanks!

This expression
res[i] = c1*(s[i] + s[i-1])/2 + c2*res[i-1] + c3*res[i-2]
says that res is the output of a linear filter (or ARMA process) with input s. Several libraries have functions for computing this. Here's how you can use the scipy function scipy.signal.lfilter.
From inspection of the recurrence relation, we get the coefficients of the numerator (b) and denominator (a) of the filter's transfer function:
b = c1 * np.array([0.5, 0.5])
a = np.array([1, -c2, -c3])
We'll also need an appropriate initial condition for lfilter to handle res[:2] == [0, k]. For this, we use scipy.signal.lfiltic:
zi = lfiltic(b, a, [k, 0], x=s[1::-1])
In the simplest case, one would call lfilter like this:
y = lfilter(b, a, s)
With an initial condition zi, we use:
y, zo = lfilter(b, a, s, zi=zi)
However, to exactly match the calculation provided in the question, we need the output y to start with [0, k]. So we'll allocate an array y, initialize the first two elements with [0, k], and assign the output of lfilter to y[2:]:
y = np.empty_like(s)
y[:2] = [0, k]
y[2:], zo = lfilter(b, a, s[2:], zi=zi)
Here's a complete script with the original loop and with lfilter:
import numpy as np
from scipy.signal import lfilter, lfiltic
c1 = 0.125
c2 = 0.5
c3 = 0.25
np.random.seed(123)
s = np.random.rand(8)
k = 3.0
# Original version (edited lightly)
res = np.zeros_like(s)
res[1] = k # k is a constant
for i in range(2, len(s)):
res[i] = c1*(s[i] + s[i-1])/2 + c2*res[i-1] + c3*res[i-2]
# Using scipy.signal.lfilter
# Coefficients of the filter's transfer function.
b = c1 * np.array([0.5, 0.5])
a = np.array([1, -c2, -c3])
# Create the initial condition of the filter such that
# y[:2] == [0, k]
zi = lfiltic(b, a, [k, 0], x=s[1::-1])
y = np.empty_like(s)
y[:2] = [0, k]
y[2:], zo = lfilter(b, a, s[2:], zi=zi)
np.set_printoptions(precision=5)
print "res:", res
print "y: ", y
The output is:
res: [ 0. 3. 1.53206 1.56467 1.24477 1.08496 0.94142 0.84605]
y: [ 0. 3. 1.53206 1.56467 1.24477 1.08496 0.94142 0.84605]
lfilter accepts an axis argument, so you can filter an array of signals with a single call. lfiltic does not have an axis argument, so setting up the initial conditions requires a loop. The following script shows an example.
import numpy as np
from scipy.signal import lfilter, lfiltic
import matplotlib.pyplot as plt
# Parameters
c1 = 0.2
c2 = 1.1
c3 = -0.5
k = 1
# Create an array of signals for the demonstration.
np.random.seed(123)
nsamples = 50
nsignals = 4
s = np.random.randn(nsamples, nsignals)
# Coefficients of the filter's transfer function.
b = c1 * np.array([0.5, 0.5])
a = np.array([1, -c2, -c3])
# Create the initial condition of the filter for each signal
# such that
# y[:2] == [0, k]
# We need a loop here, because lfiltic is not vectorized.
zi = np.empty((2, nsignals))
for i in range(nsignals):
zi[:, i] = lfiltic(b, a, [k, 0], x=s[1::-1, i])
# Create the filtered signals.
y = np.empty_like(s)
y[:2, :] = np.array([0, k]).reshape(-1, 1)
y[2:, :], zo = lfilter(b, a, s[2:], zi=zi, axis=0)
# Plot the filtered signals.
plt.plot(y, linewidth=2, alpha=0.6)
ptp = y.ptp()
plt.ylim(y.min() - 0.05*ptp, y.max() + 0.05*ptp)
plt.grid(True)
plt.show()
Plot: