bayesml.autoregressive package

bayesml.autoregressive package#

Module contents#

The linear autoregressive model with the normal-gamma prior distribution.

The stochastic data generative model is as follows:

\(d \in \mathbb{N}\): the degree of the model
\(n \in \mathbb{N}\): time index
\(x_n \in \mathbb{R}\): a data point at \(n\)
\(\boldsymbol{x}'_n := [1, x_{n-d}, x_{n-d+1}, \dots , x_{n-1}]^\top \in \mathbb{R}^{d+1}\). Here we assume \(x_n\) for \(n < 1\) is given as a initial value.
\(\boldsymbol{\theta} \in \mathbb{R}^{d+1}\): a regression coefficient parameter
\(\tau \in \mathbb{R}_{>0}\): a precision parameter of noise

\[\begin{split}p(x_n | \boldsymbol{x}'_{n-1}, \boldsymbol{\theta}, \tau) &= \mathcal{N}(x_n|\boldsymbol{\theta}^\top \boldsymbol{x}'_{n-1}, \tau^{-1}) \\ &= \sqrt{\frac{\tau}{2 \pi}} \exp \left\{ -\frac{\tau}{2} (x_n - \boldsymbol{\theta}^\top \boldsymbol{x}'_{n-1})^2 \right\}.\end{split}\]

\[\begin{split}&\mathbb{E}[ x_n | \boldsymbol{x}'_{n-1},\boldsymbol{\theta},\tau] = \boldsymbol{\theta}^{\top} \boldsymbol{x}'_{n-1}, \\ &\mathbb{V}[ x_n | \boldsymbol{x}'_{n-1},\boldsymbol{\theta},\tau ] = \tau^{-1}.\end{split}\]

The prior distribution is as follows:

\(\boldsymbol{\mu}_0 \in \mathbb{R}^{d+1}\): a hyperparameter for \(\boldsymbol{\theta}\)
\(\boldsymbol{\Lambda}_0 \in \mathbb{R}^{(d+1) \times (d+1)}\): a hyperparameter for \(\boldsymbol{\theta}\) (a positive definite matrix)
\(| \boldsymbol{\Lambda}_0 | \in \mathbb{R}\): the determinant of \(\boldsymbol{\Lambda}_0\)
\(\alpha_0 \in \mathbb{R}_{>0}\): a hyperparameter for \(\tau\)
\(\beta_0 \in \mathbb{R}_{>0}\): a hyperparameter for \(\tau\)
\(\Gamma(\cdot): \mathbb{R}_{>0} \to \mathbb{R}\): the Gamma function

\[\begin{split}p(\boldsymbol{\theta}, \tau) &= \mathcal{N}(\boldsymbol{\theta}|\boldsymbol{\mu}_0, (\tau \boldsymbol{\Lambda}_0)^{-1}) \mathrm{Gam}(\tau|\alpha_0,\beta_0)\\ &= \frac{|\tau \boldsymbol{\Lambda}_0|^{1/2}}{(2 \pi)^{(d+1)/2}} \exp \left\{ -\frac{\tau}{2} (\boldsymbol{\theta} - \boldsymbol{\mu}_0)^\top \boldsymbol{\Lambda}_0 (\boldsymbol{\theta} - \boldsymbol{\mu}_0) \right\} \frac{\beta_0^{\alpha_0}}{\Gamma (\alpha_0)} \tau^{\alpha_0 - 1} \exp \{ -\beta_0 \tau \} .\end{split}\]

\[\begin{split}\mathbb{E}[\boldsymbol{\theta}] &= \boldsymbol{\mu}_0 & \left( \alpha_0 > \frac{1}{2} \right), \\ \mathrm{Cov}[\boldsymbol{\theta}] &= \frac{\beta_0}{\alpha_0 - 1} \boldsymbol{\Lambda}_0^{-1} & (\alpha_0 > 1), \\ \mathbb{E}[\tau] &= \frac{\alpha_0}{\beta_0}, \\ \mathbb{V}[\tau] &= \frac{\alpha_0}{\beta_0^2}.\end{split}\]

The posterior distribution is as follows:

\(x^n := [x_1, x_2, \dots , x_n]^\top \in \mathbb{R}^n\): given data
\(\boldsymbol{X}_n = [\boldsymbol{x}'_1, \boldsymbol{x}'_2, \dots , \boldsymbol{x}'_n]^\top \in \mathbb{R}^{n \times (d+1)}\)
\(\boldsymbol{\mu}_n \in \mathbb{R}^{d+1}\): a hyperparameter for \(\boldsymbol{\theta}\)
\(\boldsymbol{\Lambda}_n \in \mathbb{R}^{(d+1) \times (d+1)}\): a hyperparameter for \(\boldsymbol{\theta}\) (a positive definite matrix)
\(\alpha_n \in \mathbb{R}_{>0}\): a hyperparameter for \(\tau\)
\(\beta_n \in \mathbb{R}_{>0}\): a hyperparameter for \(\tau\)

\[\begin{split}p(\boldsymbol{\theta}, \tau | x^n) &= \mathcal{N}(\boldsymbol{\theta}|\boldsymbol{\mu}_n, (\tau \boldsymbol{\Lambda}_n)^{-1}) \mathrm{Gam}(\tau|\alpha_n,\beta_n)\\ &= \frac{|\boldsymbol{\tau \Lambda}_n|^{1/2}}{(2 \pi)^{(d+1)/2}} \exp \left\{ -\frac{\tau}{2} (\boldsymbol{\theta} - \boldsymbol{\mu}_n)^\top \boldsymbol{\Lambda}_n (\boldsymbol{\theta} - \boldsymbol{\mu}_n) \right\} \frac{\beta_n^{\alpha_n}}{\Gamma (\alpha_n)} \tau^{\alpha_n - 1} \exp \{ -\beta_n \tau \} .\end{split}\]

\[\begin{split}\mathbb{E}[\boldsymbol{\theta} | x^n] &= \boldsymbol{\mu}_n & \left( \alpha_n > \frac{1}{2} \right), \\ \mathrm{Cov}[\boldsymbol{\theta} | x^n] &= \frac{\beta_n}{\alpha_n - 1} \boldsymbol{\Lambda}_n^{-1} & (\alpha_n > 1), \\ \mathbb{E}[\tau | x^n] &= \frac{\alpha_n}{\beta_n}, \\ \mathbb{V}[\tau | x^n] &= \frac{\alpha_n}{\beta_n^2},\end{split}\]

where the updating rules of the hyperparameters are

\[\begin{split}\boldsymbol{\Lambda}_n &= \boldsymbol{\Lambda}_0 + \boldsymbol{X}_n^\top \boldsymbol{X}_n,\\ \boldsymbol{\mu}_n &= \boldsymbol{\Lambda}_n^{-1} (\boldsymbol{\Lambda}_0 \boldsymbol{\mu}_0 + \boldsymbol{X}_n^\top x^n),\\ \alpha_n &= \alpha_0 + \frac{n}{2},\\ \beta_n &= \beta_0 + \frac{1}{2} \left( -\boldsymbol{\mu}_n^\top \boldsymbol{\Lambda}_n \boldsymbol{\mu}_n + (x^n)^\top x^n + \boldsymbol{\mu}_0^\top \boldsymbol{\Lambda}_0 \boldsymbol{\mu}_0 \right).\end{split}\]

The predictive distribution is as follows:

\(x_{n+1} \in \mathbb{R}\): a new data point
\(m_\mathrm{p} \in \mathbb{R}\): a parameter
\(\lambda_\mathrm{p} \in \mathbb{R}_{>0}\): a parameter
\(\nu_\mathrm{p} \in \mathbb{R}_{>0}\): a parameter

\[\mathrm{St}(x_{n+1}|m_\mathrm{p}, \lambda_\mathrm{p}, \nu_\mathrm{p}) = \frac{\Gamma (\nu_\mathrm{p}/2 + 1/2)}{\Gamma (\nu_\mathrm{p}/2)} \left( \frac{\lambda_\mathrm{p}}{\pi \nu_\mathrm{p}} \right)^{1/2} \left[ 1 + \frac{\lambda_\mathrm{p}(x_{n+1}-m_\mathrm{p})^2}{\nu_\mathrm{p}} \right]^{-\nu_\mathrm{p}/2 - 1/2}.\]

\[\begin{split}\mathbb{E}[x_{n+1} | x^n] &= m_\mathrm{p} & (\nu_\mathrm{p} > 1), \\ \mathbb{V}[x_{n+1} | x^n] &= \frac{1}{\lambda_\mathrm{p}} \frac{\nu_\mathrm{p}}{\nu_\mathrm{p}-2} & (\nu_\mathrm{p} > 2),\end{split}\]

where the parameters are obtained from the hyperparameters of the posterior distribution as follows.

\[\begin{split}m_\mathrm{p} &= \boldsymbol{\mu}_n^\top \boldsymbol{x}'_n,\\ \lambda_\mathrm{p} &= \frac{\alpha_n}{\beta_n} (1 + (\boldsymbol{x}'_n)^\top \boldsymbol{\Lambda}_n^{-1} \boldsymbol{x}'_n)^{-1},\\ \nu_\mathrm{p} &= 2 \alpha_n.\end{split}\]

class bayesml.autoregressive.GenModel(c_degree, theta_vec=None, tau=1.0, h_mu_vec=None, h_lambda_mat=None, h_alpha=1.0, h_beta=1.0, seed=None)#

Bases: Generative

The stochastic data generative model and the prior distribution.

Parameters:

c_degreeint: a positive integer.
theta_vecnumpy ndarray, optional: a vector of real numbers, which includs the constant term, by default [0.0, 0.0, … , 0.0]
taufloat, optional: a positive real number, by default 1.0
h_mu_vecnumpy ndarray, optional: a vector of real numbers, by default [0.0, 0.0, … , 0.0]
h_lambda_matnumpy ndarray, optional: a positive definate matrix, by default the identity matrix
h_alphafloat, optional: a positive real number, by default 1.0
h_betafloat, optional: a positive real number, by default 1.0
seed{None, int}, optional: A seed to initialize numpy.random.default_rng(), by default None

Methods

`gen_params`()	Generate the parameter from the prior distribution.
`gen_sample`(sample_length[, initial_values])	Generate a sample from the stochastic data generative model.
`get_constants`()	Get constants of GenModel.
`get_h_params`()	Get the hyperparameters of the prior distribution.
`get_params`()	Get the parameter of the sthocastic data generative model.
`load_h_params`(filename)	Load the hyperparameters to h_params.
`load_params`(filename)	Load the parameters saved by `save_params`.
`save_h_params`(filename)	Save the hyperparameters using python `pickle` module.
`save_params`(filename)	Save the parameters using python `pickle` module.
`save_sample`(filename, sample_length[, ...])	Save the generated sample as NumPy `.npz` format.
`set_h_params`([h_mu_vec, h_lambda_mat, ...])	Set the hyperparameters of the prior distribution.
`set_params`([theta_vec, tau])	Set the parameter of the sthocastic data generative model.
`visualize_model`([sample_length, sample_num, ...])	Visualize the stochastic data generative model and generated samples.

get_constants()#

Get constants of GenModel.

Returns:

constantsdict of {str: int}

"c_degree" : the value of self.c_degree

set_h_params(h_mu_vec=None, h_lambda_mat=None, h_alpha=None, h_beta=None)#

Set the hyperparameters of the prior distribution.

Parameters:

h_mu_vecnumpy ndarray, optional: a vector of real numbers, by default None.
h_lambda_matnumpy ndarray, optional: a positive definate matrix, by default None.
h_alphafloat, optional: a positive real number, by default None.
h_betafloat, optional: a positive real number, by default None.

get_h_params()#

Get the hyperparameters of the prior distribution.

Returns:

h_paramsdict of {str: float or numpy ndarray}

"h_mu_vec" : The value of self.h_mu_vec
"h_lambda_mat" : The value of self.h_lambda_mat
"h_alpha" : The value of self.h_alpha
"h_beta" : The value of self.h_beta

gen_params()#

Generate the parameter from the prior distribution.

The generated vaule is set at self.theta_vec and ``self.tau.

set_params(theta_vec=None, tau=None)#

Set the parameter of the sthocastic data generative model.

Parameters:

theta_vecnumpy ndarray, optional: a vector of real numbers, by default None
taufloat, optional, optional: a positive real number, by default None

get_params()#

Get the parameter of the sthocastic data generative model.

Returns:

paramsdict of {str: float or numpy ndarray}

"theta_vec" : The value of self.theta_vec.
"tau" : The value of self.tau.

gen_sample(sample_length, initial_values=None)#

Generate a sample from the stochastic data generative model.

Parameters:

sample_lengthint: A positive integer
initial_valulesnumpy ndarray, optional: 1 dimensional float array whose size coincide with self.c_degree, by default None.

Returns:

xnumpy ndarray: 1 dimensional float array whose size is sammple_length.

save_sample(filename, sample_length, initial_values=None)#

Save the generated sample as NumPy .npz format.

It is saved as a NpzFile with keyword: “x”.

Parameters:

filenamestr: The filename to which the sample is saved. .npz will be appended if it isn’t there.
sample_lengthint: A positive integer
initial_valulesnumpy ndarray, optional: 1 dimensional float array whose size coincide with self.c_degree, by default None.

See also

numpy.savez_compressed

visualize_model(sample_length=50, sample_num=5, initial_values=None)#

Visualize the stochastic data generative model and generated samples.

Parameters:

sample_lengthint, optional: A positive integer, by default 50
sample_numint, optional: A positive integer, by default 5
initial_valulesnumpy ndarray, optional: 1 dimensional float array whose size coincide with self.c_degree, by default None.

Examples

>>> import numpy as np
>>> from bayesml import autoregressive
>>> model = autoregressive.GenModel(c_degree=1,theta_vec=np.array([0,1]))
>>> model.visualize_model()
theta_vec:[0,1]
tau:1.0

class bayesml.autoregressive.LearnModel(c_degree, h0_mu_vec=None, h0_lambda_mat=None, h0_alpha=1.0, h0_beta=1.0)#

Bases: Posterior, PredictiveMixin

The posterior distribution and the predictive distribution.

Parameters:

c_degreeint: a positive integer.
h0_mu_vecnumpy ndarray, optional: a vector of real numbers, by default [0.0, 0.0, … , 0.0]
h0_lambda_matnumpy ndarray, optional: a positive definate matrix, by default the identity matrix
h0_alphafloat, optional: a positive real number, by default 1.0
h0_betafloat, optional: a positive real number, by default 1.0

Attributes:

hn_mu_vecnumpy ndarray: a vector of real numbers
hn_lambda_matnumpy ndarray: a positive definate matrix
hn_alphafloat: a positive real number
hn_betafloat: a positive real number
p_mfloat: a positive real number
p_lambdafloat: a positive real number
p_nufloat: a positive real number

Methods

`calc_pred_dist`(x)	Calculate the parameters of the predictive distribution.
`estimate_params`([loss])	Estimate the parameter of the stochastic data generative model under the given criterion.
`get_constants`()	Get constants of LearnModel.
`get_h0_params`()	Get the initial values of the hyperparameters of the posterior distribution.
`get_hn_params`()	Get the hyperparameters of the posterior distribution.
`get_p_params`()	Get the parameters of the predictive distribution.
`load_h0_params`(filename)	Load the hyperparameters to h0_params.
`load_hn_params`(filename)	Load the hyperparameters to hn_params.
`make_prediction`([loss])	Predict a new data point under the given criterion.
`overwrite_h0_params`()	Overwrite the initial values of the hyperparameters of the posterior distribution by the learned values.
`pred_and_update`(x[, loss])	Predict a new data point and update the posterior sequentially.
`predict_interval`([credibility])	Credible interval of the prediction.
`reset_hn_params`()	Reset the hyperparameters of the posterior distribution to their initial values.
`save_h0_params`(filename)	Save the hyperparameters using python `pickle` module.
`save_hn_params`(filename)	Save the hyperparameters using python `pickle` module.
`set_h0_params`([h0_mu_vec, h0_lambda_mat, ...])	Set initial values of the hyperparameter of the posterior distribution.
`set_hn_params`([hn_mu_vec, hn_lambda_mat, ...])	Set updated values of the hyperparameter of the posterior distribution.
`update_posterior`(x[, padding])	Update the hyperparameters of the posterior distribution using traning data.
`visualize_posterior`()	Visualize the posterior distribution for the parameter.

get_constants()#

Get constants of LearnModel.

Returns:

constantsdict of {str: int}

"c_degree" : the value of self.c_degree

set_h0_params(h0_mu_vec=None, h0_lambda_mat=None, h0_alpha=None, h0_beta=None)#

Set initial values of the hyperparameter of the posterior distribution.

Note that the parameters of the predictive distribution are also calculated from self.h0_mu_vec, slef.h0_lambda_mat, self.h0_alpha and self.h0_beta.

Parameters:

h0_mu_vecnumpy ndarray, optional: a vector of real numbers, by default None.
h0_lambda_matnumpy ndarray, optional: a positive definate matrix, by default None.
h0_alphafloat, optional: a positive real number, by default None.
h0_betafloat, optional: a positive real number, by default None.

get_h0_params()#

Get the initial values of the hyperparameters of the posterior distribution.

Returns:

h0_paramsdict of {str: float or numpy ndarray}

"h0_mu_vec" : The value of self.h0_mu_vec
"h0_lambda_mat" : The value of self.h0_lambda_mat
"h0_alpha" : The value of self.h0_alpha
"h0_beta" : The value of self.h0_beta

get_hn_params()#

Get the hyperparameters of the posterior distribution.

Returns:

hn_paramsdict of {str: float or numpy ndarray}

"hn_mu_vec" : The value of self.hn_mu_vec
"hn_lambda_mat" : The value of self.hn_lambda_mat
"hn_alpha" : The value of self.hn_alpha
"hn_beta" : The value of self.hn_beta

set_hn_params(hn_mu_vec=None, hn_lambda_mat=None, hn_alpha=None, hn_beta=None)#

Set updated values of the hyperparameter of the posterior distribution.

Note that the parameters of the predictive distribution are also calculated from self.hn_mu_vec, slef.hn_lambda_mat, self.hn_alpha and self.hn_beta.

Parameters:

hn_mu_vecnumpy ndarray, optional: a vector of real numbers, by default None.
hn_lambda_matnumpy ndarray, optional: a positive definate matrix, by default None.
hn_alphafloat, optional: a positive real number, by default None.
hn_betafloat, optional: a positive real number, by default None.

update_posterior(x, padding=None)#

Update the hyperparameters of the posterior distribution using traning data.

Parameters:

xnumpy ndarray: 1 dimensional float array
paddingstr, optional: Padding option for data values at negative time points. Default is None, in which case the first self.c_degree values of X are used as initial values. If “zeros” is given, the zero vector is used as a initial value.

estimate_params(loss='squared')#

Estimate the parameter of the stochastic data generative model under the given criterion.

Note that the criterion is applied to estimating theta_vec and tau independently. Therefore, a tuple of the student’s t-distribution and the gamma distribution will be returned when loss=”KL”

Parameters:

lossstr, optional: Loss function underlying the Bayes risk function, by default “squared”. This function supports “squared”, “0-1”, “abs”, and “KL”.

Returns:

Estimatestuple of {numpy ndarray, float, None, or rv_frozen}

theta_vec_hat : the estimate for theta_vec
tau_hat : the estimate for tau

The estimated values under the given loss function. If it is not exist, None will be returned. If the loss function is “KL”, the posterior distribution itself will be returned as rv_frozen object of scipy.stats.

bayesml.autoregressive package

Contents

bayesml.autoregressive package#

Module contents#