log(VA) = gamma - (1/eta)log[alphaL^(-eta) + betaK^(-eta)]
I'm trying to estimate the above function with nonlinear least squares. I am using 3 different packages (Scipy-minimize, Scipy-curve_fit and lmfit - Model) for this but I find different parameter results in each one. I can't understand why. I would be very grateful if anyone can help with a solution or offer a different solution method.
SCIPY-MINIMIZE
import numpy as np
from scipy.optimize import minimize, curve_fit
from lmfit import Model, Parameters
L = np.array([0.299, 0.295, 0.290, 0.284, 0.279, 0.273, 0.268, 0.262, 0.256, 0.250])
K = np.array([2.954, 3.056, 3.119, 3.163, 3.215, 3.274, 3.351, 3.410, 3.446, 3.416])
VA = np.array([0.919, 0.727, 0.928, 0.629, 0.656, 0.854, 0.955, 0.981, 0.908, 0.794])
def f(param):
gamma = param[0]
alpha = param[1]
beta = param[2]
eta = param[3]
VA_est = gamma - (1/eta)*np.log(alpha*L**-eta + beta*K**-eta)
return np.sum((np.log(VA) - VA_est)**2)
bnds = [(1, np.inf), (0,1),(0,1),(-1, np.inf)]
x0 = (1,0.01,0.98, 1)
con = {"type":"eq", "fun":c}
result = minimize(f, x0, bounds = bnds)
print(result.fun)
print(result.message)
print(result.x[0],result.x[1],result.x[2],result.x[3])
SCIPY-MINIMIZE - OUT
0.30666062040617503
CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL
1.0 0.5587147011643757 0.9371430857380681 5.873041615873815
SCIPY-CURVE_FIT
def f(X, gamma, alpha, beta, eta):
L,K = X
return gamma - (1/eta) * np.log(alpha*L**-eta + beta*K**-eta)
p0 = 1,0.01,0.98, 1
res, cov = curve_fit(f, (L, K), np.log(VA), p0, bounds = ((1,0,0,-1),(np.inf,1,1,np.inf)) )
gamma, alpha, beta, eta = res[0],res[1],res[2],res[3]
gamma, alpha, beta, eta
SCIPY-CURVE_FIT - OUT
(1.000000000062141,
0.26366547263939205,
0.9804436474926481,
13.449747863921704)
LMFIT-MODEL
def f(x, gamma, alpha, beta, eta):
L = x[0]
K = x[1]
return gamma - (1/eta)*np.log(alpha*L**-eta + beta*K**-eta)
fmodel = Model(f)
params = Parameters()
params.add('gamma', value = 1, vary=True, min = 1)
params.add('alpha', value = 0.01, vary=True, max = 1, min = 0)
params.add('beta', value = 0.98, vary=True, max = 1, min = 0)
params.add('eta', value = 1, vary=True, min = -1)
result = fmodel.fit(np.log(VA), params, x=(L,K))
print(result.fit_report())
LMFIT-MODEL - OUT
[[Model]]
Model(f)
[[Fit Statistics]]
# fitting method = leastsq
# function evals = 103
# data points = 10
# variables = 4
chi-square = 0.31749840
reduced chi-square = 0.05291640
Akaike info crit = -26.4986758
Bayesian info crit = -25.2883354
## Warning: uncertainties could not be estimated:
gamma: at initial value
gamma: at boundary
alpha: at boundary
[[Variables]]
gamma: 1.00000000 (init = 1)
alpha: 1.3245e-13 (init = 0.01)
beta: 0.20130064 (init = 0.98)
eta: 447.960413 (init = 1)
A fitting algorithm always seeks for a local minimizer of the underlying least-squares problem. Note that your problem is convex but not strictly convex. Consequently, there's no unique global minimizer. But each local minimizer is a global one. By evaluating the first function f for each found solution, we can observe that they all have the same objective function value. Hence, each solution is a global minimizer.
Why does each method find a different minimizer? The reason is simple. Each one uses a different algorithm to solve the underlying nonlinear optimization problem, e.g. scipy.optimize.minimize uses the 'L-BFGS-B' algorithm while scipy.optimize.curve_fit uses scipy.optimize.least_squares with the Trust Region Reflective algorithm ('TRF'). In short, you can only expect the same solution for different algorithms for a strictly convex problem.
Related
I have a complicated curve fitting function:
def corr_function(tau: np.ndarray, BG: float, avg_C: float, vz: float):
wxc = 8.3
wy = 2.5
wz = 3.35
D = 4.4e1
return 1/((math.pi)**(3/2)*wxc*wy*wz*avg_C)*(1 + 4*D*tau/(wxc**2))**(-1/2)*(1 + 4*D*tau/(wy**2))**(-1/2)*(1 + 4*D*tau/(wz**2))**(-1/2)*np.exp(-((vz*tau)**2/(wz**2 + 4*D*tau))) + BG
I tried to fit this with scipy:
popt, pcov = curve_fit(corr_function, tau, corr, [0, 1e-12, 2e5])
and lmfit
model = Model(corr_function, independent_vars=['tau'])
result = model.fit(
corr,
tau=tau,
BG=Parameter('BG', value=0, min=0),
avg_C=Parameter('avg_C', value=1e-12, min=0),
vz=Parameter('vz', value=2e5, min=0),
)
And while the scipy converges to a proper answer (blue), the lmfit doesn't (orange), where lmfit parameters don't change really at all during fitting
[[Fit Statistics]]
# fitting method = leastsq
# function evals = 61
# data points = 400
# variables = 3
chi-square = 1.5370e+12
reduced chi-square = 3.8714e+09
Akaike info crit = 8833.74620
Bayesian info crit = 8845.72059
## Warning: uncertainties could not be estimated:
BG_guess: at boundary
avg_C_guess: at initial value
avg_C_guess: at boundary
[[Variables]]
BG: 0.00000000 (init = 0)
avg_C: 3.9999e-12 (init = 4e-12)
vz: 8831416.63 (init = 200000)
I think I need lmfit to sample a larger parameter space (or more iterations), anyone know how to do this?
Also, note, I need to the input parameters to be static (can't bring them closer to proper fit), as I'll need to automate fitting for large parameter spaces
I am trying to minimize a loss function over an Ordinary differential equation (ODE) problem, using python.
This loss function is meant to retrieve the best coefficient rates of a user defined ODE.
Despite my efforts and examples found in the internet, all results ended up with ODEs whose solutions returned coefficient values which were outside of their respective bounds.
I know that this can happen for several reasons: choice of minimizer, bad ODE definition, integration step-size, among other things.
Nevertheless, my major problem is to find an example where the minimizer can support integration "Breaks" in the ODE integration.
For example, if I am interested in a simple ODE model, for example the Prey-Predator ODE (Lotka-Volterra), one rule that must be set is that no populations can achieve negative values.
Therefore, if the ODE minimizer for some given reason returns a negative value for any of the two populations modeled (Prey and Predator), the minimizer must stop its integration.
Nevertheless, as it appears, this kind of rule is not supported by common minimizations, at least for the Minimizer objects of the lmfit that I tested.
Even with the bounding rules setted for each Parameter object, the Minimizers continue to interpolate the data beyond the given thresholds (i.e.: negative populations).
For example:
Let's assume that I want to find the best coefficient rates of a Prey-Predator ODE with respect to some empirical data.
In this case, if I use the python's lmfit library for my problem, I could do something like below:
from lmfit import Parameters, report_fit, Minimizer
import numpy as np
import matplotlib.pyplot as plt
from scipy.integrate import ode
end = '\n'*2 + '-'*50 + '\n'*2
def ode_f(t, xs, ps):
"""Lotka-Volterra predator-prey model."""
if isinstance(ps, Parameters):
Prey_Natality = ps[r'Prey_Natality'].value
Predation_rate = ps['Predation_rate'].value
Predator_Natality = ps['Predator_Natality'].value
Predator_Mortality = ps['Predator_Mortality'].value
Prey_natural_mortality = ps['Prey_natural_mortality'].value
else:
(Prey_Natality, Predation_rate, Predator_Natality,
Predator_Mortality, Prey_natural_mortality) = ps
Prey, Predator = xs
dPrey_dt = ( + Prey_Natality*Prey
- Predation_rate*(Prey)*Predator
- Prey_natural_mortality*Prey)
dPred_dt = ( + Predation_rate* (Prey ) *Predator
+ Predator_Natality*Predator
- Predator_Mortality*Predator)
return [dPrey_dt, dPred_dt]
def solout(y):
if np.any( y <=0):
return -1 # stop integration
else:
return 0
def ode_solver(t,
x0,
ps,
mode='vode',
nsteps=500,
method='bdf'):
"""
Solution to the ODE x'(t) = f(t,x,k) with initial condition x(0) = x0
"""
r = ode(ode_f).set_integrator(mode,
nsteps=nsteps,
method=method)
t0 = t.min()
tmax = t.max()
dt = np.diff(t)[0]
r.set_initial_value(x0, t0).set_f_params(ps)
y = []
times = []
integration_time = r.t
while r.successful() and integration_time < tmax:
r.integrate(integration_time + dt)
integration_time = r.t
yi = r.y
y.append(yi)
times.append(integration_time)
if solout(yi) == -1:
print('Time stoped at: {0}'.format(integration_time))
break
return (np.array(y)).astype(float), times
def ODE_solver_residual_evaluator(ps, ts, data):
x0 = [ps['Prey_Pop'].value, ps['Predator_Pop'].value]
model, times = ode_solver(ts, x0, ps)
# if data.shape[0] <= model.shape[0]:
# data.resize((model.shape[0],data.shape[1]),refcheck=False)
# else:
# model.resize((data.shape[0],data.shape[1]),refcheck=False)
return ( model[:len(data )] - data[:len(model)] )
def residual_dim_reducer(residual_array):
return np.square(residual_array).sum()
if '__main__' == __name__:
########
dt = 10**(-4)
t = np.arange(0, 100, dt)
Prey_initial_pop = 1200
Predator_initial_pop = 50
x0 = np.array([Prey_initial_pop,
Predator_initial_pop])
Prey_Natality = 2.6
Predation_rate = 0.12
Predator_Natality = 0.401
Predator_Mortality = 0.0025
Prey_natural_mortality = 0.001
true_params = np.array((Prey_Natality,
Predation_rate,
Predator_Natality,
Predator_Mortality,
Prey_natural_mortality))
data, times = ode_solver(t, x0, true_params)
data += np.random.lognormal(size=data.shape)*0.5
Param_Names = ['Prey_Natality',
'Predation',
'Predator_Natality',
'Predator_Mortality',
'Prey_natural_mortality']
Populations = ['Prey population',
'Predator population']
for i in range(data.shape[1]):
plt.plot(times, np.real(data[:,i]), 'o', label=r'original {0}'.format(Populations[i]))
plt.legend()
plt.show()
import os
to_save = os.path.join(os.getcwd(), 'original data.png')
plt.savefig(to_save)
print('Creating the minizer object', end=end)
################3
# set parameters incluing bounds
params = Parameters()
params.add('Prey_Pop', value= 100, min=1, max=600)
params.add('Predator_Pop', value=10, min=1, max=400)
params.add('Prey_Natality', value=0.03, min=0.000001, max=3.5)
params.add('Predation_rate', value=0.02, min=0.00003, max=3.5)
params.add('Predator_Natality', value=0.0004, min=0.00001, max=3.2)
params.add('Predator_Mortality', value=0.003, min=0.000001, max=3.2)
params.add('Prey_natural_mortality', value=0.001, min=0.0000001, max=0.2)
fitter = Minimizer(ODE_solver_residual_evaluator,
params,
fcn_args=(t, data),
iter_cb=None,
scale_covar=True,
nan_policy='propagate',
reduce_fcn=residual_dim_reducer,
calc_covar=True,
)
fitted_ODE = fitter.minimize(method='Nelder-Mead')
Optimum_params = fitted_ODE.params.valuesdict()
x0_optimum = np.array([Optimum_params.pop('Prey_Pop'),
Optimum_params.pop('Predator_Pop')])
Y_fitted, times_fitted = ode_solver(t, x0_optimum, Optimum_params.values())
Param_Names = list(params.keys())
print(end, 'Param_Names: {0}'.format(Param_Names))
##################
data = data[:len(Y_fitted)]
Y_fitted = Y_fitted[:len(data)]
from sklearn.metrics import (explained_variance_score,
r2_score,
mean_squared_error)
explained_variance_score = explained_variance_score(Y_fitted, data)
R2 = r2_score(Y_fitted, data)
RMSE = np.sqrt(mean_squared_error(Y_fitted, data))
print('Explained variance: {0} \n R2: {1} \n RMSE: {2}'.format(explained_variance_score,
R2, RMSE))
print(end, 'Fitting by Minizer is complete', end=end)
# display fitted statistics
report_fit(fitted_ODE)
print(end)
######################3
fig2 = plt.figure()
n_ref_markers = 12
markers_on = np.linspace(0, data[:,i].size-1, n_ref_markers).astype(int).tolist()
for i in range(Y_fitted.shape[1]):
plt.plot(times_fitted[:len(times)],
Y_fitted[:len(times),i],
'-',
linewidth=1.1,
label=r'fitted {0} '.format(Param_Names[i]))
plt.plot(np.arange(len(data)), data[:,i], 'o',
markevery=markers_on,
label=r'original {0}'.format(Param_Names[i]))
fig2.legend()
fig2.show()
The returned values are:
Two figures:
A) the original data
B) the original plus the ODE's modeled data according to the best coefficient rates that lmfit returned
A table of returns containing fitting measurements and parameters statistical descriptions.
Returned output:
--------------------------------------------------
Fitting by Minizer is complete
--------------------------------------------------
Models fitting error:
Explained variance: -90.1682809468072
R2: -3125.4358694840084
RMSE: 785.9581933129715
--------------------------------------------------
[[Fit Statistics]]
# fitting method = Nelder-Mead
# function evals = 66437
# data points = 364
# variables = 7
chi-square = 2.2485e+08
reduced chi-square = 629842.649
Akaike info crit = 4867.50583
Bayesian info crit = 4894.78590
## Warning: uncertainties could not be estimated:
[[Variables]]
Prey_Pop: 117.479436 +/- 16.3998313 (13.96%) (init = 100)
Predator_Pop: 397.552948 +/- nan (nan%) (init = 10)
Prey_Natality: 1.05567443 +/- nan (nan%) (init = 0.03)
Predation_rate: 3.46543190 +/- 0.05124925 (1.48%) (init = 0.02)
Predator_Natality: 0.48528830 +/- nan (nan%) (init = 0.0004)
Predator_Mortality: 2.76733581 +/- 0.06777831 (2.45%) (init = 0.003)
Prey_natural_mortality: 0.03928761 +/- 0.00503378 (12.81%) (init = 0.001)
[[Correlations]] (unreported correlations are < 0.100)
C(Predation_rate, Prey_natural_mortality) = -0.596
C(Prey_Pop, Predator_Mortality) = 0.179
C(Predation_rate, Predator_Mortality) = 0.141
C(Prey_Pop, Prey_natural_mortality) = 0.127
C(Predator_Mortality, Prey_natural_mortality) = -0.112
This problem came from a didactic example extracted from here:
Any help on the subject would be appreciated.
Sincerely,
I am trying to fit a model function to a curve using the lmfit module.
The curve that I am fitting is set up as follows:
e(x) = exp(-(x-X)/x0) for x larger or equal than X, 0 otherwise.
G(x) = (1/sqrt(2*pi)*sigma) * exp(-x^2/2*sigma^2)
The model fit M(x) = E * conv(e,G)(x) + B
Where e is a truncated exponential, G is a gaussian, and E and B are constants. The operator between e and G is a convolution.
When I try to fit this function to my data I get a good fit. However, the fit is very sensitive to my initial value that I provide for X. This is also reflected in the uncertainty in the parameters:
[[Model]]
((Model(totemiss) * (Model(exptruncated) <function convolve at 0x7f139e2dcde8> Model(gaussian))) + Model(background))
[[Fit Statistics]]
# fitting method = leastsq
# function evals = 67
# data points = 54
# variables = 5
chi-square = 120558969110355112544642583094864038386991104.00000
reduced chi-square = 2460387124701124853181382654239391973638144.00000
Akaike info crit = 5275.63336
Bayesian info crit = 5285.57828
[[Variables]]
E: 9.7316e+28 +/- 2.41e+33 (2475007.74%) (init= 1.2e+29)
x0: 5.9420e+06 +/- 9.52e+04 (1.60%) (init= 5000000)
X: 4.9049e+05 +/- 1.47e+11 (29978575.17%) (init= 100000)
sigma: 2.6258e+06 +/- 5.74e+04 (2.19%) (init= 2000000)
center: 0 (fixed)
amplitude: 1 (fixed)
B: 3.9017e+22 +/- 3.75e+20 (0.96%) (init= 4.5e+22)
[[Correlations]] (unreported correlations are < 0.100)
C(E, X) = -1.000
C(sigma, B) = -0.429
C(x0, sigma) = -0.283
C(x0, B) = -0.266
C(E, x0) = -0.105
C(x0, X) = 0.105
I suspect this has something to do due to the correlation between E and X being -1.00, which does not make any sense. I am trying to find out why I get this error and I believe it might be in the definition of the model:
def exptruncated(x, x0, X):
return np.exp(-(x-X)/x0)* (x > X)
#Define convolution operator
def convolve(arr, kernel):
npts = min(len(arr), len(kernel))
pad = np.ones(npts)
tmp = np.concatenate((pad*arr[0], arr, pad*arr[-1]))
out = np.convolve(tmp, kernel, mode='valid')
noff = int((len(out) - npts)/2)
return out[noff:noff+npts]
#Constant value for total emissions#
def totemiss(x,E):
return E
#Constant value for background value
def background(x,B):
return B
# create Composite Model using the custom convolution operator
# M(x) = E + conv(exp,gauss) + B
mod = Model(totemiss)* CompositeModel(Model(exptruncated), Model(gaussian), convolve) + Model(background)
mod.set_param_hint('x0',value=50*1e5,min=0,max=60*1e5)
mod.set_param_hint('amplitude',value=1.0)
mod.set_param_hint('center',value=0.0)
mod.set_param_hint('sigma',value=20*1e5,min=0,max=100*1e5)
mod.set_param_hint('X',value=1.0*1e5,min=0, max=5.0*1e5)
mod.set_param_hint('B',value=0.45*1e23,min=0.3*1e23,max=1.0*1e23)
mod.set_param_hint('E',value=1.2*1e29,min=1.2*1e26,max=1.0*1e32)
pars = mod.make_params()
pars['amplitude'].vary = False
pars['center'].vary = False
result = mod.fit(y, params=pars, x=x)
comps = result.eval_components(x=x)
Although I believe the model is the reason I am not able to find where the error comes from. Perhaps somebody of you can help me out!
Why not just remove E from the model -- the X parameter is serving as a constant offset.
I'd also advise having parameters that are more reasonably scaled, closer to order of unity (roughly 1e-6 to 1e6, say). You can add scales of 1e10 and so on as needed in the model calculation, but it generally helps the calculations of covariance (used to determine how to update values in the fit) to have parameters more uniformly scaled.
From the scipy minimize documentation:
args : tuple, optional Extra arguments passed to the objective function and its derivatives
Great. And that's pretty much all it says about the args parameter.
So I have this code that produces a Holt-Winters forecast courtesy of grisha.org. In the blog post, the author determined that the optimal values for alpha, beta and gamma were 0.716, 0.029, 0.993 respectively.
I suspect I can use scipy's minimize function to solve for these three variables by minimizing the sum of squared errors. That's what the sse function does below:
def sse(alpha, beta, gamma):
pred = triple_exponential_smoothing(series, 12, alpha, beta, gamma, 24)
return sum((pred[:series.shape[0]] - series)**2)
My problem is, my objective function sse takes three parameters (aka: multiple parameters). What I want to be able to do is pass sse into minimize and let minimize tell me what the optimal values for alpha, beta and gamma are. All three parameters are bound between 0 and 1. The "optimal" value will obviously be the three values that achieve the smallest sse value.
Can this be achieved with scipy.optimize.minimize?
minimize(sse, [0.5, 0.5, 0.5], args=(beta, gamma),
bounds=[(0, 1), (0, 1), (0, 1)], method='SLSQP')
Here's my full code:
import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import minimize
def initial_trend(series, slen):
sum = 0.0
for i in range(slen):
sum += float(series[i+slen] - series[i]) / slen
return sum / slen
def initial_seasonal_components(series, slen):
seasonals = {}
season_averages = []
n_seasons = int(len(series)/slen)
# compute season averages
for j in range(n_seasons):
season_averages.append(sum(series[slen*j:slen*j+slen])/float(slen))
# compute initial values
for i in range(slen):
sum_of_vals_over_avg = 0.0
for j in range(n_seasons):
sum_of_vals_over_avg += series[slen*j+i]-season_averages[j]
seasonals[i] = sum_of_vals_over_avg/n_seasons
return seasonals
def triple_exponential_smoothing(series, slen, alpha, beta, gamma, n_preds):
result = []
seasonals = initial_seasonal_components(series, slen)
for i in range(len(series)+n_preds):
if i == 0: # initial values
smooth = series[0]
trend = initial_trend(series, slen)
result.append(series[0])
continue
if i >= len(series): # we are forecasting
m = len(series) - i + 1
result.append((smooth + m*trend) + seasonals[i%slen])
else:
val = series[i]
last_smooth, smooth = smooth, alpha*(val-seasonals[i%slen]) + (1-alpha)*(smooth+trend)
trend = beta * (smooth-last_smooth) + (1-beta)*trend
seasonals[i%slen] = gamma*(val-smooth) + (1-gamma)*seasonals[i%slen]
result.append(smooth+trend+seasonals[i%slen])
return np.array(result)
def sse(alpha, beta, gamma):
pred = triple_exponential_smoothing(series, 12, alpha, beta, gamma, 24)
return sum((pred[:series.shape[0]] - series)**2)
series = np.array([30,21,29,31,40,48,53,47,37,39,31,29,17,9,20,24,27,35,41,38,
27,31,27,26,21,13,21,18,33,35,40,36,22,24,21,20,17,14,17,19,
26,29,40,31,20,24,18,26,17,9,17,21,28,32,46,33,23,28,22,27,
18,8,17,21,31,34,44,38,31,30,26,32])
#pred = triple_exponential_smoothing(series, 12, 0.716, 0.029, 0.993, 24)
pred = triple_exponential_smoothing(series, 12, 0.1, 0.9, 0.1, 24)
fig, ax = plt.subplots(1,1)
ax.plot(pred, 'r:', label='Forecast')
ax.plot(series, 'k-', label='Actual', lw=1)
ax.legend(loc='best')
plt.show()
Before alpha, beta, gamma parameter optimization
After alpha, beta, gamma parameter optimization
Say I have a random collection of (X,Y) points:
import pymc as pm
import numpy as np
import matplotlib.pyplot as plt
import scipy
x = np.array(range(0,50))
y = np.random.uniform(low=0.0, high=40.0, size=200)
y = map((lambda a: a[0] + a[1]), zip(x,y))
plt.scatter(x,y)
and that I fit simple linear regression:
std = 20.
tau=1/(std**2)
alpha = pm.Normal('alpha', mu=0, tau=tau)
beta = pm.Normal('beta', mu=0, tau=tau)
sigma = pm.Uniform('sigma', lower=0, upper=20)
y_est = alpha + beta * x
likelihood = pm.Normal('y', mu=y_est, tau=1/(sigma**2), observed=True, value=y)
model = pm.Model([likelihood, alpha, beta, sigma, y_est])
mcmc = pm.MCMC(model)
mcmc.sample(40000, 15000)
How can I get the distribution or the statistics of y_est[0], y_est[1], y_est[2].. (note that these variables correspond to the y values estimated for each input x value.
In PyMC 2, if you are interested in the trace of a deterministic, you should wrap the deterministic in a Lambda object (or decorate a function with #deterministic). In your case, this would be:
y_est = Lambda('y_est', lambda a=alpha, b=beta: a + b * x)
You should then be able to call the summary method or plot the node, just like a Stochastic.
BTW, you do not need to instantiate a Model object, as MCMC already does that for you. All you need is:
mcmc = pm.MCMC([likelihood, alpha, beta, sigma, y_est])
or even more concisely:
mcmc = pm.MCMC(vars())
Following #Chris' advice, the following works:
x = pm.Uniform('x', lower=xmin, upper=xmax)
alpha = pm.Normal('alpha', mu=0, tau=tau)
beta = pm.Normal('beta', mu=0, tau=tau)
sigma = pm.Uniform('sigma', lower=0, upper=20)
# The deterministic:
y_gen = pm.Lambda('y_gen', lambda a=alpha, x=x, b=beta: a + b * x)
And then draw samples from it as follows:
mcmc = pm.MCMC([x, y_gen])
mcmc.sample(n_total_samples, n_burn_in)
x_trace = mcmc.trace('x')[:]
y_trace = mcmc.trace('y_gen')[:]