Related
I am in the process of "vectorizing" a pet project of mine. The function 'propagateXi' propagates (solves) a time-varying, linear system of equations and answers to ~20% of the cost of the project.
The sizes vary a lot between instances of the problem. A "typical" set would be: N = 501, n = 4, s = 3.
Here is a simplification of the code:
import numpy
class Propagator():
def __init__(self,mode):
# lots of code here
pass
def propagateXi(self,Xi,terms):
""" This function propagates the linear system, called many times with different initial conditions.
The initial conditions were previously set.
The "arc" represents a kind of "realization";
the arcs are essentially independent from one another.
Sizes: "vector dimensions": 2*n, "time dimension": N, "arcs dimension": s
- Xi (N, 2*n, s) stores the (vector) state in each time and each "arc".
- self.MainPropMat (N, 2*n, 2*n, s) stores the 2n x 2n matrices
- terms (N, 2*n, s) stores the "forcing terms" of the equation.
The equation would be something like:
\xi_{k+1, arc} = MainPropMat_{k, arc} * \xi_{k, arc} + terms_{k, arc} \forall k, arc
"""
for k in range(self.N - 1):
Xi[k+1,:,:] = numpy.einsum('ijs,js->is', self.MainPropMat[k, :, :, :], Xi[k, :, :]) +\
terms[k, :, :]
return Xi
I have tried to re-interpret the problem as the multiplication of a huge 2nN x 2nN matrix by a long 2nN column containing the initial conditions and the forcing terms. In these conditions, the code was something like this:
import numpy
class Propagator():
def __init__(self,mode):
# lots of code here
self.prepareBigMat()
def prepareBigMat(self)
"""
Prepare the "big matrix" for Xi propagation. Called only once.
This function assembles the "big matrix" (s x 2*n*N x 2*n*N !) for propagating
the Xi differential equation in a single step.
:return: None
"""
n2 = 2 * self.n
BigMat = numpy.empty((self.s,self.N * n2,self.N * n2))
I = numpy.eye(n2*self.N)
for arc in range(self.s):
BigMat[arc,:,:] = I
for k in range(1,self.N):
kn2 = k * n2
BigMat[:, kn2:(kn2+n2) , 0:kn2] = numpy.einsum('ijs,sjk->sik',
self.MainPropMat[k-1,:,:,:],
BigMat[:,(kn2-n2):k*n2, 0:kn2])
self.BigMat = BigMat
def propagateXi(self,Xi,terms):
n2 = n * 2
# Assemble the big column
BigCol = numpy.empty((n2 * N, s))
# Initial conditions
BigCol[:n2, :] = Xi[0, :, :]
# Forcing terms
BigCol[n2:, :] = terms.reshape((n2 * (N - 1), s))
# Perform the multiplication and reshaping of the Xi array
Xi = numpy.einsum('sij,js->is', self.BigMat, BigCol).reshape((N, n2, s))
return Xi
The second version does precisely the same thing as the first one. Sadly, this second version takes almost 5x the running time of the previous one. My guess is that the cost of assembly of the Big Matrix is too great...
Any ideas?
Thanks!
I am learning Python and Probabilities.
I have as a given that the expression:
c = n! /((n-1)!1!) + 2*n! /((n-2)*2!) + 3*n!/((n-3)*3!) + ...+ n*n!/((n-n)*n!
where 0! = 1 and ! signifies 'factorial', i.e. n! = 1*2*3*...*(n-1)*n.
is equal to:
(a + n^b)*2^(c*n + d). (^ signifies exponent)
My goal is to determine the parameters a, b, c, d using 'brute force'.
I calculated using the above formula c for n=3 (12), n=4 (38), n=5 (80), n=6 (192), n=7 (448).
Then I expressed the parameters as ratios of two integers: i.e.
a = a1/a2, b=b1/b2, c=c1/c2, d=d1/d2.
Finally I defined the following function:
def com():
parms = []
for a1 in range(-10, 10):
for a2 in range(1,11):
for b1 in range(-10,10):
for b2 in range(1, 11):
for c1 in range(-10,10):
for c2 in range(1, 11):
for d1 in range(-10 , 10):
for d2 in range(1 , 11):
a = a1/a2
b = b1/b2
c = c1/c2
d = d1/d2
cr1 = ( 12 == (a + 3**b)*2**(c*3+d) )
cr2 = ( 38 == (a + 4**b)*2**(c*4+d) )
cr3 = ( 80 == (a + 5**b)*2**(c*5+d) )
cr4 = ( 192 == (a + 6**b)*2**(c*6+d) )
cr5 = ( 448 == (a + 7**b)*2**(c*7+d) )
criterion = cr1 & cr2 & cr3 & cr4 & cr5
if criterion == 1 :
parms = [a, b, c, d]
break
return parms
However, my function returns an empty list.
Could you explain that? Do you have any suggestions on how to achieve my objective?
Your advice will be appreciated.
Firstly, let's look at some bugs in your code:
criterion = cr1 & cr2 & cr3 & cr4 & cr5
This is performing bitwise operations on the values. You probably wanted:
criterion = cr1 and cr2 and cr3 and cr4 and cr5
Python also provides an all function, which you could check if all things are True:
criterion = all([cr1,cr2,cr3,cr4,cr5])
Now, let's look at your if statement:
if criterion == 1 :
Since you want to know if criterion is True or False, you can simply use if criterion:.
Now lastly, this approach is unlikely to ever be True. Take this line for example:
cr1 = ( 12 == (a + 3**b)*2**(c*3+d) )
Those numbers would have to add up to exactly 12, else it will be False, and then you will get an empty list.
Also, computers can't do decimal maths accurately.
To solve the actual values, you must use parameter substitution. That's not a programming question, but maths and programming go together well, so here's a starter:
12 = (a + 3**b)*2**(c*3+d)
12/(a+3**b) = 2**(c*3+d)
... and so on. Get a in terms of b, c,and d, then use your cr2 to substitute in a for the numbers you have, and get b in terms of c and d.
Repeat a few more times, and you have numbers for the four values.
You can use scipy curvefit and use non-linear least squares to fit a function, f, to data and find the parameters (a,b,c,d) values. The example data that you provided fit to the function (a + n**b)+2**(c*n + d) better. Notice that best fit for the values of a,b,c,d are floating point numbers, not integers.
import numpy as np
from scipy.optimize import curve_fit
xdata = np.array(range(3,8))
ydata = np.array([12,38,80,192,448])
def func(n, a, b, c, d):
return (a + n**b)+2**(c*n + d)
popt, pcov = curve_fit(func, xdata, ydata, p0=(1,1,1,1))
a, b, c, d = popt
print a, b, c, d # learnt parameters
# -5.62374782967 1.79345876905 1.29232902743 -0.328778229316
import matplotlib.pyplot as plt
plt.scatter(xdata, ydata)
plt.plot(xdata, func(xdata, a, b, c, d), '-r', label='np.poly')
plt.show()
In the above fitted curve, blue points are the data points and red line is the fitted function with the learnt parameters, notice that the function is continuous and defined for all values of n.
With brains instead of Python:
You recognize a modified Binomial sum, Sum k.Cnk instead of Sum Cnk.
Then notice that
k.n!/(k!(n-k)!) = n!/((k-1)!(n-k)!) = n.(n-1)!/(k-1)!(n-1-k+1)!) = n.Cn-1,k+1.
So your sum will be n.2^(n-1).
Before rushing into brute force, it may be wiser to attempt to estimate a tight range for the parameters.
The first values of the sum are
1, 4, 12, 32, 80, 192, 448, 1024, 2304, 5120, 11264
Taking the pairwise ratios, you observe that they get closer and closer to 2.
As the ratio (a + (n+1)^b)/(a + n^b) will tend to 1 and the ratio 2^(c(n+1)+d)/2^(cn+d) will tend to 2^c, you are hinted that c is probably close to 1. You can check this estimate by trying larger and larger values of n (For instance, n=1000 yields the ratio 2.002002... = 2^1.00144...).
Then to reduce the growth rate of the expression to make it more tractable, it can be interesting to look at the values of Sum/2^n, and we get
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11
Hum, hum.
Unfortunately, this case is too easy for a proper demonstration of the method. The general idea is to observe the asymptotic behavior to get a rough estimate of some of the parameters. And when you have such a rough estimate (in a small range), then you can somehow cancel its effect to better see the effect of the other ones.
For exhaustive searches, understanding the ranges is important.
My apologize for the lengthy title. I have been working on a project for a a while and i'm in a rut for a certain part in my code. Ill do my best to be thorough.
I have an numpy array of masses, M, of size and shape 167197.
## Non-constant
M = data['m200'] # kg // Mass of dark matter haloes
R = [] # Km // Radius of sphere
for masses in M:
R.append(((3*masses)/(RHO_C*4*(3.14))**(1.0/3.0)))
I have fitting function with independent values of k that are part of my question. k is defined value in my code.
def T(k): # Fitting Function // Assuming a lambdaCDM model
q = k/((OMEGA_M)*H**2)*((T_CMB)/27)**2
L = np.log(euler+1.84*q)
C = 14.4 + 325/(1+60.5*q**1.11)
return L/(L+C*q**2)
##############################################################################
def P(k): # Linear Power Spectrum
A = 0.75 # LambdaCDM Power Normalization
n = 0.95 # current constraints from WMAP+LSS
return A*k**n*T(k)**2
* For the actual problem *
I have a Fourier transfrom W(kR)
def W(R):# Fourier Transfrom of Top Hat function
return (3*(np.sin(k*R)-(k*R)*np.cos(k*R)))/(k*R)**(3)
W_a = []
for radii in R:
W_a.append(W(radii))
In this condition, i'm treating R as the independent value instead of kR combined
printing the length of W_a gives me the exact same size as mu numpy array, so all is well.
This function will play a part for a integral along with the is included in this function of sigma
def sigma(R): # Mass Varience
k1 = lambda k: k**2*P(k)*W(R)**2
norm1 = 1/(2*np.pi**2)
return (integrate.quad(k1, 0, np.Inf))
sigma_a = []
for radii in R:
sigma_a.extend(sigma(radii))
The integral will create a tuple, of course. But for each value inside R. I'm wanting to create a list, or an array. So, when using .extend(), the length of my array is now doubled with a length of now 334394.
How do i correct it to where the integral evaluates each R in W(kR) returning an array of the same size, 167197?
First just a Python note:
R = [] # Km // Radius of sphere
for masses in M:
R.append(((3*masses)/(RHO_C*4*(3.14))**(1.0/3.0)))
can be expressed as:
R = [((3*masses)/(RHO_C*4*(3.14))**(1.0/3.0)) for masses in M]
In:
return (integrate.quad(k1, 0, np.Inf))
the outer set of () doesn't make a difference.
return integrate.quad(k1, 0, np.Inf)
should return the same thing.
Now where does the doubling come from? In the quad docs we see it returns 2 values, the integeral and an error term. That's shown as a tuple in some examples, but it is also unpacked in others:
y, err = integrate.quad(f, 0, 1, args=(3,))
If you want just integral, and not err, you could index, integate...()[0].
sigma_a = []
for radii in R:
sigma_a.append(sigma(radii)[0])
or
sigma_a = [sigma(radii)[0] for radii in R]
or
def sigma1(R): # Mass Varience
k1 = lambda k: k**2*P(k)*W(R)**2
norm1 = 1/(2*np.pi**2)
y, err = integrate.quad(k1, 0, np.Inf)
return y # return just the integral
sigma_a = [sigma1(radii) for radii in R]
If you want to collect both y and err, but in separate lists, use zip* to repack them (something like the numpy transpose).
ll = [sigma(radii) for radii in R]
# [(y0,err0),(y1,err1), ...]
ys, errs = zip(*ll)
I have some large arrays each with i elements, call them X, Y, Z, for which I need to find some values a, b--where a and b are real numbers between 0 and 1--such that, for the following functions,
r = X - a*Y - b*Z
r_av = Sum(r)/i
rms = Sum((r - r_av)^2), summing over the i pixels
I want to minimize the rms. Basically I'm looking to minimize the scatter in r, and thus need to find the right a and b to do that. So far I have thought to do this in nested loops in one of two ways: either 1)just looping through a range of possible a,b and then selecting out the smallest rms, or 2)inserting a while statement so that the loop will terminate once rms stops decreasing with decreasing a,b for instance. Here's some pseudocode for these:
1) List
for a = 1
for b = 1
calculate m
b = b - .001
a = a - .001
loop 1000 times
sort m values, from smallest
print (a,b) corresponding to smallest m
2) Terminate
for a = 1
for b = 1
calculate m
while m > previous step,
b = b - .001
a = a - .001
Is one of these preferable? Or is there yet another, better way to go about this? Any tips would be greatly appreciated.
There is already a handy formula for least squares fitting.
I came up with two different ways to solve your problem.
For the first one, consider the matrix K:
L = len(X)
K = np.identity(L) - np.ones((L, L)) / L
In your case, A and B are defined as:
A = K.dot(np.array([Y, Z]).transpose())
B = K.dot(np.array([X]).transpose())
Apply the formula to find C that minimizes the error A * C - B:
C = np.linalg.inv(np.transpose(A).dot(A))
C = C.dot(np.transpose(A)).dot(B)
Then the result is:
a, b = C.reshape(2)
Also, note that numpy already provides linalg.lstsq that does the exact same thing:
a, b = np.linalg.lstsq(A, B)[0].reshape(2)
A simpler way is to define A as:
A = np.array([Y, Z, [1]*len(X)]).transpose()
Then solve it against X to get the coefficients and the mean:
a, b, mean = np.linalg.lstsq(A, X)[0]
If you need a proof of this result, have a look at this post.
Example:
>>> import numpy as np
>>> X = [5, 7, 9, 5]
>>> Y = [2, 0, 4, 1]
>>> Z = [7, 2, 4, 6]
>>> A = np.array([Y, Z, [1] * len(X)]).transpose()
>>> a, b, mean = np.linalg.lstsq(A, X)[0]
>>> print(a, b, mean)
0.860082304527 -0.736625514403 8.49382716049
edit: I'm not looking for you to debug this code. If you are familiar with this well-known algorithm, then you may be able to help. Please note that the algorithm produces the coefficients correctly.
This code for cubic spline interpolation is producing linear splines and I can't seem to figure out why (yet). The algorithm comes from Burden's Numerical Analysis, which is just about identical to the pseudo code here, or you can find that book from a link in the comments (see chapter 3, it's worth having anyway). The code is producing the correct coefficients; I believe that I am misunderstanding the implementation. Any feedback is greatly appreciated. Also, i'm new to programming, so any feedback on how bad my coding is also welcome. I tried uploading pics of the linear system in terms of h, a, and c, but as a new user i can not. If you want a visual of the tridiagonal linear system that the algorithm solves, and which is set up by the var alpha, see the link in the comments for the book, see chap 3. The system is strictly diagonally dominant, so we know there exists a unique solution c0,...,cn. Once we know the ci values, the other coefficients follow.
import matplotlib.pyplot as plt
# need some zero vectors...
def zeroV(m):
z = [0]*m
return(z)
#INPUT: n; x0, x1, ... ,xn; a0 = f(x0), a1 =f(x1), ... , an = f(xn).
def cubic_spline(n, xn, a, xd):
"""function cubic_spline(n,xn, a, xd) interpolates between the knots
specified by lists xn and a. The function computes the coefficients
and outputs the ranges of the piecewise cubic splines."""
h = zeroV(n-1)
# alpha will be values in a system of eq's that will allow us to solve for c
# and then from there we can find b, d through substitution.
alpha = zeroV(n-1)
# l, u, z are used in the method for solving the linear system
l = zeroV(n+1)
u = zeroV(n)
z = zeroV(n+1)
# b, c, d will be the coefficients along with a.
b = zeroV(n)
c = zeroV(n+1)
d = zeroV(n)
for i in range(n-1):
# h[i] is used to satisfy the condition that
# Si+1(xi+l) = Si(xi+l) for each i = 0,..,n-1
# i.e., the values at the knots are "doubled up"
h[i] = xn[i+1]-xn[i]
for i in range(1, n-1):
# Sets up the linear system and allows us to find c. Once we have
# c then b and d follow in terms of it.
alpha[i] = (3./h[i])*(a[i+1]-a[i])-(3./h[i-1])*(a[i] - a[i-1])
# I, II, (part of) III Sets up and solves tridiagonal linear system...
# I
l[0] = 1
u[0] = 0
z[0] = 0
# II
for i in range(1, n-1):
l[i] = 2*(xn[i+1] - xn[i-1]) - h[i-1]*u[i-1]
u[i] = h[i]/l[i]
z[i] = (alpha[i] - h[i-1]*z[i-1])/l[i]
l[n] = 1
z[n] = 0
c[n] = 0
# III... also find b, d in terms of c.
for j in range(n-2, -1, -1):
c[j] = z[j] - u[j]*c[j+1]
b[j] = (a[j+1] - a[j])/h[j] - h[j]*(c[j+1] + 2*c[j])/3.
d[j] = (c[j+1] - c[j])/(3*h[j])
# This is my only addition, which is returning values for Sj(x). The issue I'm having
# is related to this implemention, i suspect.
for j in range(n-1):
#OUTPUT:S(x)=Sj(x)= aj + bj(x - xj) + cj(x - xj)^2 + dj(x - xj)^3; xj <= x <= xj+1)
return(a[j] + b[j]*(xd - xn[j]) + c[j]*((xd - xn[j])**2) + d[j]*((xd - xn[j])**3))
For the bored, or overachieving...
Here is code for testing, the interval is x: [1, 9], y:[0, 19.7750212]. The test function is xln(x), so we start 1 and increase by .1 up to 9.
ln = []
ln_dom = []
cub = []
step = 1.
X=[1., 9.]
FX=[0, 19.7750212]
while step <= 9.:
ln.append(step*log(step))
ln_dom.append(step)
cub.append(cubic_spline(2, x, fx, step))
step += 0.1
...and for plotting:
plt.plot(ln_dom, cub, color='blue')
plt.plot(ln_dom, ln, color='red')
plt.axis([1., 9., 0, 20], 'equal')
plt.axhline(y=0, color='black')
plt.axvline(x=0, color='black')
plt.show()
Ok, got this working. The problem was in my implementation. I got it working with a different approach, where the splines are constructed individually instead of continuously. This is fully functioning cubic spline interpolation by method of first constructing the coefficients of the spline polynomials (which is 99% of the work), then implementing them. Obviously this is not the only way to do it. I may work on a different approach and post that if there is interest. One thing that would clarify the code would be an image of the linear system that is solved, but i can't post pics until my rep gets up to 10. If you want to go deeper into the algorithm, see the text book link in the comments above.
import matplotlib.pyplot as plt
from pylab import arange
from math import e
from math import pi
from math import sin
from math import cos
from numpy import poly1d
# need some zero vectors...
def zeroV(m):
z = [0]*m
return(z)
#INPUT: n; x0, x1, ... ,xn; a0 = f(x0), a1 =f(x1), ... , an = f(xn).
def cubic_spline(n, xn, a):
"""function cubic_spline(n,xn, a, xd) interpolates between the knots
specified by lists xn and a. The function computes the coefficients
and outputs the ranges of the piecewise cubic splines."""
h = zeroV(n-1)
# alpha will be values in a system of eq's that will allow us to solve for c
# and then from there we can find b, d through substitution.
alpha = zeroV(n-1)
# l, u, z are used in the method for solving the linear system
l = zeroV(n+1)
u = zeroV(n)
z = zeroV(n+1)
# b, c, d will be the coefficients along with a.
b = zeroV(n)
c = zeroV(n+1)
d = zeroV(n)
for i in range(n-1):
# h[i] is used to satisfy the condition that
# Si+1(xi+l) = Si(xi+l) for each i = 0,..,n-1
# i.e., the values at the knots are "doubled up"
h[i] = xn[i+1]-xn[i]
for i in range(1, n-1):
# Sets up the linear system and allows us to find c. Once we have
# c then b and d follow in terms of it.
alpha[i] = (3./h[i])*(a[i+1]-a[i])-(3./h[i-1])*(a[i] - a[i-1])
# I, II, (part of) III Sets up and solves tridiagonal linear system...
# I
l[0] = 1
u[0] = 0
z[0] = 0
# II
for i in range(1, n-1):
l[i] = 2*(xn[i+1] - xn[i-1]) - h[i-1]*u[i-1]
u[i] = h[i]/l[i]
z[i] = (alpha[i] - h[i-1]*z[i-1])/l[i]
l[n] = 1
z[n] = 0
c[n] = 0
# III... also find b, d in terms of c.
for j in range(n-2, -1, -1):
c[j] = z[j] - u[j]*c[j+1]
b[j] = (a[j+1] - a[j])/h[j] - h[j]*(c[j+1] + 2*c[j])/3.
d[j] = (c[j+1] - c[j])/(3*h[j])
# Now that we have the coefficients it's just a matter of constructing
# the appropriate polynomials and graphing.
for j in range(n-1):
cub_graph(a[j],b[j],c[j],d[j],xn[j],xn[j+1])
plt.show()
def cub_graph(a,b,c,d, x_i, x_i_1):
"""cub_graph takes the i'th coefficient set along with the x[i] and x[i+1]'th
data pts, and constructs the polynomial spline between the two data pts using
the poly1d python object (which simply returns a polynomial with a given root."""
# notice here that we are just building the cubic polynomial piece by piece
root = poly1d(x_i,True)
poly = 0
poly = d*(root)**3
poly = poly + c*(root)**2
poly = poly + b*root
poly = poly + a
# Set up our domain between data points, and plot the function
pts = arange(x_i,x_i_1, 0.001)
plt.plot(pts, poly(pts), '-')
return
If you want to test it, here's some data you can use to get started, which comes from the
function 1.6e^(-2x)sin(3*pi*x) between 0 and 1:
# These are our data points
x_vals = [0, 1./6, 1./3, 1./2, 7./12, 2./3, 3./4, 5./6, 11./12, 1]
# Set up the domain
x_domain = arange(0,2, 1e-2)
fx = zeroV(10)
# Defines the function so we can get our fx values
def sine_func(x):
return(1.6*e**(-2*x)*sin(3*pi*x))
for i in range(len(x_vals)):
fx[i] = sine_func(x_vals[i])
# Run cubic_spline interpolant.
cubic_spline(10,x_vals,fx)
Comments on your coding style:
Where are your comments and documentation? At the very least, provide function documentation so that people can tell how your function is supposed to be used.
Instead of:
def cubic_spline(xx,yy):
Please write something like:
def cubic_spline(xx, yy):
"""function cubic_spline(xx,yy) interpolates between the knots
specified by lists xx and yy. The function returns the coefficients
and ranges of the piecewise cubic splines."""
You can make lists of repeated elements by using the * operator on a list.
Like this:
>>> [0] * 10
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
So that your zeroV function can be replaced by [0] * m.
Just don't do this with mutable types! (especially lists).
>>> inner_list = [1,2,3]
>>> outer_list = [inner_list] * 3
>>> outer_list
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
>>> inner_list[0] = 999
>>> outer_list
[[999, 2, 3], [999, 2, 3], [999, 2, 3]] # wut
Math should probably be done using numpy or scipy.
Apart from that, you should read Idiomatic Python by David Goodger.