I want to create a function that calculates and returns the projection of a vector x on a vector b as well as the reconstruction error.
My code is the following:
def reconstruction_error(x, b):
'''The function calculates the projection and reconstruction error
from projecting a vector x onto a vector b'''
b = np.matrix(b)
x_projection_on_b = (b.T # b/ float((b#b.T))) # x
reconstruction_error = (x - x_projection_on_b) # (x - x_projection_on_b).T
return( x_projection_on_b, float(reconstruction_error))
However the reconstruction error is not correct. E.g.,
x = np.array([1,1,1])
b = np.array([5, 10, 10])
a, error = reconstruction_error(x, b)
a
matrix([[0.55555556, 1.11111111, 1.11111111]])
error
0.2222222222222222
Not sure about terminology, but if "reconstruction error" is length of "rejection vector" (original vector minus its projection), then you would have:
import numpy as np
from numpy.linalg import norm
a = np.array([1,1,1])
b = np.array([5, 10, 10])
def projection(x, on):
return np.dot(x, on) / np.dot(on, on) * on
def rejection(x, on):
return x - projection(x, on)
def reconstruction_error(x, on):
return norm(rejection(x, on))
>>> reconstruction_error(a, b)
0.4714045207910317
you are correct in your approach
reconstruction_error = (x - x_projection_on_b) # (x - x_projection_on_b).T
However, what you have calculated above is squared norm.
Apply -> np.sqrt(reconstruction_error) to get the right answer
Related
I'm trying to use scipy curve_fit to capture the value of a0 parameter. As of now, it is not changing (always comes out as 1):
X = [[1,2,3],[4,5,6]]
def func(X, a0, c):
x1 = X[0]; x2 = X[1]
a = x1*x2
result = np.where(a(a<a0), -c*(a + np.sqrt(x2)), -c*x1)
return result
Popt, Cov = scipy.curve_fit(func, X, y)
a0, c = Popt
Predicted = func(X, a0, c) # a0 and c are constants
I get the values for c, which is a scalar, without any problem. I can't explain why a0 (also a scalar) is always 1, and I am not sure how to fix it. I did see elsewhere on SO that np.where can be used the way I have used it here, but apparently not for curve_fit function. Maybe I need to use a different method of optimization, and I'd like some pointers to do this using scipy methods.
Edit: I tried the construct suggested by Brad, but that's not it.
Updated!
This should work. note that the a variable is a vector in this example of length 3 because it is computed by the element wise multiplication of the first and second elements of X which is a 2x3 matrix. Therefore a0 can either be a scalar or a vector of length 3 and c can also be a scalar or a vector of length 3.
import numpy as np
X = np.array([[1, 2, 3], [4, 5, 6]])
a0 = np.array([8,25,400])
# a0 = 2
# Code works whether C is scalar or a matrix since it can be broadcast to matrix a below.
# c = 3 # Uncomment this for scalar
c = np.array([8, 12, 2000]) # Element wise
def func(X, a0, c):
x = X[0]
y = X[1]
a = x * y
print(a.shape)
result = np.where(a < a0, c * (a + np.sqrt(y)), c * x)
return result
func(X, a0, c)
This is a minimum amount of code that works. Notice I removed the y>0 and defined a to be the same size as c. Now you get the correct insertions because the first parameter of np.where is now the same size as the second and third parameters. Before (x<a) & (y>0) always evaluated to True or False and that is a scalar in this context. If a was a N dimensional array you would have received a ValueError because the operands could not be broadcast together
import numpy as np
c = np.array([[22,34],[33,480]])
def func(X, a):
x = X[0]; y = X[1]
return np.where(c[(x<a)], -c*(a + np.sqrt(y)), -c*x)
X = [25, 600]
a = np.array([[2,14],[33,22]])
func(X,a)
This also works if c is a constant and a was the array you wanted manipulated
import numpy as np
c = 2
def func(X, a):
x = X[0]; y = X[1]
return np.where(a[(x<a)], -c*(a + np.sqrt(y)), -c*x)
X = [25, 600]
a = np.array([[2,14],[33,22]])
func(X,a)
I am trying to maximize a target function f(x) with function scipy.optimize.minimum. But it usually takes 4-5 hrs to run the code because the function f(x) involves a lot of computation of complex matrix. To improve its speed, I want to use gpu. And I've already tried tensorflow package. Since I use numpy to define f(x), I have to convert it into tensorflow's format. However, it doesn't support the computation of complex matrix. What else package or means I can use? Any suggestions?
To specific my problem, I will show calculate scheme below:
Calculate the expectation :
-where H=x*H_0, x is the parameter
Let \phi go through the dynamics of Schrödinger equation
-Different H is correspond to a different \phi_end. Thus, parameter x determines the expectation
Change x, calculate the corresponding expectation
Find a specific x that minimize the expectation
Here is a simple example of part of my code:
import numpy as np
import cmath
from scipy.linalg import expm
import scipy.optimize as opt
# create initial complex matrixes
N = 2 # Dimension of matrix
H = np.array([[1.0 + 1.0j] * N] * N) # a complex matrix with shape(N, N)
A = np.array([[0.0j] * N] * N)
A[0][0] = 1.0 + 1j
# calculate the expectation
def value(phi):
exp_H = expm(H) # put the matrix in the exp function
new_phi = np.linalg.linalg.matmul(exp_H, phi)
# calculate the expectation of the matrix
x = np.linalg.linalg.matmul(H, new_phi)
expectation = np.inner(np.conj(phi), x)
return expectation
# Contants
tmax = 1
dt = 0.1
nstep = int(tmax/dt)
phi_init = [1.0 + 1.0j] * N
# 1st derivative of Schrödinger equation
def dXdt(t, phi, H): # 1st derivative of the function
return -1j * np.linalg.linalg.matmul(H, phi)
def f(X):
phi = [[0j] * N] * nstep # store every time's phi
phi[0] = phi_init
# phi go through the dynamics of Schrödinger equation
for i in range(nstep - 1):
phi[i + 1] = phi[i] - dXdt(i * dt, X[i] * H, phi[i]) * dt
# calculate the corresponding value
f_result = value(phi[-1])
return f_result
# Initialize the parameter
X0 = np.array(np.ones(nstep))
results = opt.minimize(f, X0) # minimize the target function
opt_x = results.x
PS:
Python Version: 3.7
Operation System: Win 10
I would like to be able to apply a generic function on either scalars numpy 1-D arrays, o numpy 2-D arrays.
The example in point is
def stress2d_lefm_cyl(KI, r, qdeg) :
"""Compute stresses in Mode I around a 2D crack, according to LEFM
q should be input in degrees"""
sfactor = KI / sqrt(2*pi*r)
q = radians(qdeg)
q12 = q/2; q32 = 3*q/2;
sq12 = sin(q12); cq12 = cos(q12);
sq32 = sin(q32); cq32 = cos(q32);
af11 = cq12 * (1 - sq12*sq32); af22 = cq12 * (1 + sq12*sq32);
af12 = cq12 * sq12 * cq32
return sfactor * np.array([af11, af22, af12])
def stress2d_lefm_rect(KI, x, y) :
"""Compute stresses in Mode I around a 2D crack, according to LEFM
"""
r = sqrt(x**2+y**2) <-- Error line
q = atan2(y, x)
return stress2d_lefm_cyl(KI, r, degrees(q))
delta = 0.5
x = np.arange(-10.0, 10.01, delta)
y = np.arange(0.0, 10.01, delta)
X, Y = np.meshgrid(x, y)
KI = 1
# I want to pass a scalar KI, and either scalar, 1D, or 2D arrays for X,Y (of the same shape, of course)
Z = stress2d_lefm_rect(KI, X, Y)
TypeError: only size-1 arrays can be converted to Python scalars
(I mean to use this for a contour plot).
If I now change to
def stress2d_lefm_rect(KI, x, y) :
"""Compute stresses in Mode I around a 2D crack, according to LEFM
"""
r = lambda x,y: x**2 + y**2 <-- Now this works
q = lambda x,y: atan2(y, x) <-- Error line
return stress2d_lefm_cyl(KI, r(x,y), degrees(q(x,y)))
Z = stress2d_lefm_rect(KI, X, Y)
TypeError: only size-1 arrays can be converted to Python scalars
which boils down to
x = np.array([1.0, 2, 3, 4, 5])
h = lambda x,y: atan2(y,x) <-- Error
print(h(0,1)) <-- Works
print(h(x, x)) <-- Error
1.5707963267948966
TypeError: only size-1 arrays can be converted to Python scalars
A "similar" question was posted, Most efficient way to map function over numpy array
The differences are:
1. I have to (or possibly more) arguments (x,y), which should have the same shape.
2. I am combining also with a scalar argument (KI).
3. atan2 seems to be less "tolerant" than **2. I mean to work with a generic function.
4. I am chaining two functions.
Can this be worked out?
Perhaps point 2 can be overcome by multiplying the result somewhere else.
You should use numpy to apply your function to every element of an array.
Ex :
import numpy as np
np.sqrt(np.square(x) + np.square(y))
I've written some beginner code to calculate the co-efficients of a simple linear model using the normal equation.
# Modules
import numpy as np
# Loading data set
X, y = np.loadtxt('ex1data3.txt', delimiter=',', unpack=True)
data = np.genfromtxt('ex1data3.txt', delimiter=',')
def normalEquation(X, y):
m = int(np.size(data[:, 1]))
# This is the feature / parameter (2x2) vector that will
# contain my minimized values
theta = []
# I create a bias_vector to add to my newly created X vector
bias_vector = np.ones((m, 1))
# I need to reshape my original X(m,) vector so that I can
# manipulate it with my bias_vector; they need to share the same
# dimensions.
X = np.reshape(X, (m, 1))
# I combine these two vectors together to get a (m, 2) matrix
X = np.append(bias_vector, X, axis=1)
# Normal Equation:
# theta = inv(X^T * X) * X^T * y
# For convenience I create a new, tranposed X matrix
X_transpose = np.transpose(X)
# Calculating theta
theta = np.linalg.inv(X_transpose.dot(X))
theta = theta.dot(X_transpose)
theta = theta.dot(y)
return theta
p = normalEquation(X, y)
print(p)
Using the small data set found here:
http://www.lauradhamilton.com/tutorial-linear-regression-with-octave
I get the co-efficients: [-0.34390603; 0.2124426 ] using the above code instead of: [24.9660; 3.3058]. Could anyone help clarify where I am going wrong?
You can implement normal equation like below:
import numpy as np
X = 2 * np.random.rand(100, 1)
y = 4 + 3 * X + np.random.randn(100, 1)
X_b = np.c_[np.ones((100, 1)), X] # add x0 = 1 to each instance
theta_best = np.linalg.inv(X_b.T.dot(X_b)).dot(X_b.T).dot(y)
X_new = np.array([[0], [2]])
X_new_b = np.c_[np.ones((2, 1)), X_new] # add x0 = 1 to each instance
y_predict = X_new_b.dot(theta_best)
y_predict
This assumes X is an m by n+1 dimensional matrix where x_0 always = 1 and y is a m-dimensional vector.
import numpy as np
step1 = np.dot(X.T, X)
step2 = np.linalg.pinv(step1)
step3 = np.dot(step2, X.T)
theta = np.dot(step3, y) # if y is m x 1. If 1xm, then use y.T
Your implementation is correct. You've only swapped X and y (look closely how they define x and y), that's why you get a different result.
The call normalEquation(y, X) gives [ 24.96601443 3.30576144] as it should.
Here is the normal equation in one line:
theta = np.dot(np.linalg.inv(np.dot(X.T,X)),np.dot(X.T,Y))
I am trying to evaluate the density of multivariate t distribution of a 13-d vector. Using the dmvt function from the mvtnorm package in R, the result I get is
[1] 1.009831e-13
When i tried to write the function by myself in Python (thanks to the suggestions in this post:
multivariate student t-distribution with python), I realized that the gamma function was taking very high values (given the fact that I have n=7512 observations), making my function going out of range.
I tried to modify the algorithm, using the math.lgamma() and np.linalg.slogdet() functions to transform it to the log scale, but the result I got was
8.97669876e-15
This is the function that I used in python is the following:
def dmvt(x,mu,Sigma,df,d):
'''
Multivariate t-student density:
output:
the density of the given element
input:
x = parameter (d dimensional numpy array or scalar)
mu = mean (d dimensional numpy array or scalar)
Sigma = scale matrix (dxd numpy array)
df = degrees of freedom
d: dimension
'''
Num = math.lgamma( 1. *(d+df)/2 ) - math.lgamma( 1.*df/2 )
(sign, logdet) = np.linalg.slogdet(Sigma)
Denom =1/2*logdet + d/2*( np.log(pi)+np.log(df) ) + 1.*( (d+df)/2 )*np.log(1 + (1./df)*np.dot(np.dot((x - mu),np.linalg.inv(Sigma)), (x - mu)))
d = 1. * (Num - Denom)
return np.exp(d)
Any ideas why this functions does not produce the same results as the R equivalent?
Using as x = (0,0) produces similar results (up to a point, die to rounding) but with x = (1,1)1 I get a significant difference!
I finally managed to 'translate' the code from the mvtnorm package in R and the following script works without numerical underflows.
import numpy as np
import scipy.stats
import math
from math import lgamma
from numpy import matrix
from numpy import linalg
from numpy.linalg import slogdet
import scipy.special
from scipy.special import gammaln
mu = np.array([3,3])
x = np.array([1, 1])
Sigma = np.array([[1, 0], [0, 1]])
p=2
df=1
def dmvt(x, mu, Sigma, df, log):
'''
Multivariate t-student density. Returns the density
of the function at points specified by x.
input:
x = parameter (n x d numpy array)
mu = mean (d dimensional numpy array)
Sigma = scale matrix (d x d numpy array)
df = degrees of freedom
log = log scale or not
'''
p = Sigma.shape[0] # Dimensionality
dec = np.linalg.cholesky(Sigma)
R_x_m = np.linalg.solve(dec,np.matrix.transpose(x)-mu)
rss = np.power(R_x_m,2).sum(axis=0)
logretval = lgamma(1.0*(p + df)/2) - (lgamma(1.0*df/2) + np.sum(np.log(dec.diagonal())) \
+ p/2 * np.log(math.pi * df)) - 0.5 * (df + p) * math.log1p((rss/df) )
if log == False:
return(np.exp(logretval))
else:
return(logretval)
print(dmvt(x,mu,Sigma,df,True))
print(dmvt(x,mu,Sigma,df,False))