bayesml.normal package

bayesml.normal package#

Module contents#

The normal distribution with the normaml-gamma prior distribution.

The stochastic data generative model is as follows:

\(x \in \mathbb{R}\): a data point
\(\mu \in \mathbb{R}\): a mean parameter
\(\tau \in \mathbb{R}_{>0}\): a precision parameter

\[\begin{split}p(x | \mu, \tau) &= \mathcal{N}(x|\mu,\tau^{-1})\\ &= \sqrt{\frac{\tau}{2\pi}} \exp \left\{ -\frac{\tau}{2}(x-\mu)^2 \right\},\end{split}\]

\[\begin{split}&\mathbb{E}[x | \mu, \tau] = \mu, \\ &\mathbb{V}[x | \mu, \tau] = \tau^{-1}.\end{split}\]

The prior distribution is as follows:

\(m_0 \in \mathbb{R}\): a hyperparameter
\(\kappa_0 \in \mathbb{R}_{>0}\): a hyperparameter
\(\alpha_0 \in \mathbb{R}_{>0}\): a hyperparameter
\(\beta_0 \in \mathbb{R}_{>0}\): a hyperparameter
\(\Gamma ( \cdot )\): the gamma function

\[\begin{split}p(\mu, \tau) &= \mathcal{N}(\mu|m_0,(\kappa_0 \tau)^{-1}) \mathrm{Gam}(\tau | \alpha_0, \beta_0) \\ &= \sqrt{\frac{\kappa_0 \tau}{2\pi}} \exp \left\{ -\frac{\kappa_0 \tau}{2}(\mu-m_0)^2 \right\} \frac{\beta_0^{\alpha_0}}{\Gamma (\alpha_0)} \tau^{\alpha_0 - 1} \exp \{ -\beta_0 \tau \},\end{split}\]

\[\begin{split}\mathbb{E}[\mu] &= m_0 & \left( \alpha_0 > \frac{1}{2} \right), \\ \mathbb{V}[\mu] &= \frac{\beta_0}{\kappa_0 (\alpha_0 - 1)} & (\alpha_0 > 1), \\ \mathbb{E}[\tau] &= \frac{\alpha_0}{\beta_0}, \\ \mathbb{V}[\tau] &= \frac{\alpha_0}{\beta_0^2}.\end{split}\]

The posterior distribution is as follows:

\(x^n = (x_1, x_2, \dots , x_n) \in \mathbb{R}^n\): given data
\(m_n \in \mathbb{R}\): a hyperparameter
\(\kappa_n \in \mathbb{R}_{>0}\): a hyperparameter
\(\alpha_n \in \mathbb{R}_{>0}\): a hyperparameter
\(\beta_n \in \mathbb{R}_{>0}\): a hyperparameter
\(\Gamma ( \cdot )\): the gamma function

\[\begin{split}p(\mu, \tau | x^n) &= \mathcal{N}(\mu|m_n,(\kappa_n \tau)^{-1}) \mathrm{Gam}(\tau | \alpha_n, \beta_n) \\ &= \sqrt{\frac{\kappa_n \tau}{2\pi}} \exp \left\{ -\frac{\kappa_n \tau}{2}(\mu-m_n)^2 \right\} \frac{\beta_n^{\alpha_n}}{\Gamma (\alpha_n)} \tau^{\alpha_n - 1} \exp \{ -\beta_n \tau \},\end{split}\]

\[\begin{split}\mathbb{E}[\mu | x^n] &= m_n & \left( \alpha_n > \frac{1}{2} \right), \\ \mathbb{V}[\mu | x^n] &= \frac{\beta_n}{\kappa_n (\alpha_n - 1)} & (\alpha_n > 1), \\ \mathbb{E}[\tau | x^n] &= \frac{\alpha_n}{\beta_n}, \\ \mathbb{V}[\tau | x^n] &= \frac{\alpha_n}{\beta_n^2},\end{split}\]

where the updating rule of the hyperparameters is

\[\begin{split}\bar{x} &= \frac{1}{n} \sum_{i=1}^n x_i, \\ m_n &= \frac{\kappa_0 m_0 + n \bar{x}}{\kappa_0 + n}, \\ \kappa_n &= \kappa_0 + n, \\ \alpha_n &= \alpha_0 + \frac{n}{2}, \\ \beta_n &= \beta_0 + \frac{1}{2} \left( \sum_{i=1}^n (x_i - \bar{x})^2 + \frac{\kappa_0 n}{\kappa_0 + n} (\bar{x} - m_0)^2 \right).\end{split}\]

The predictive distribution is as follows:

\(x_{n+1} \in \mathbb{R}\): a new data point
\(\mu_\mathrm{p} \in \mathbb{R}\): the hyperparameter of the predictive distribution
\(\lambda_\mathrm{p} \in \mathbb{R}_{>0}\): the hyperparameter of the predictive distribution
\(\nu_\mathrm{p} \in \mathbb{R}_{>0}\): the hyperparameter of the predictive distribution

\[\begin{split}p(x_{n+1} | x^{n} ) &= \mathrm{St}(x_{n+1} | \mu_\mathrm{p}, \lambda_\mathrm{p}, \nu_\mathrm{p}) \\ &= \frac{\Gamma (\nu_\mathrm{p} / 2 + 1/2 )}{\Gamma (\nu_\mathrm{p} / 2)} \left( \frac{\lambda_\mathrm{p}}{\pi \nu_\mathrm{p}} \right)^{1/2} \left( 1 + \frac{\lambda_\mathrm{p} (x_{n+1} - \mu_\mathrm{p})^2}{\nu_\mathrm{p}} \right)^{-\nu_\mathrm{p}/2 - 1/2},\end{split}\]

\[\begin{split}\mathbb{E}[x_{n+1} | x^n] &= \mu_\mathrm{p} & (\nu_\mathrm{p} > 1), \\ \mathbb{V}[x_{n+1} | x^n] &= \frac{1}{\lambda_\mathrm{p}} \frac{\nu_\mathrm{p}}{\nu_\mathrm{p}-2} & (\nu_\mathrm{p} > 2),\end{split}\]

where the parameters are obtained from the hyperparameters of the posterior distribution as follows:

\[\begin{split}\mu_\mathrm{p} &= \mu_n, \\ \lambda_\mathrm{p} &= \frac{\kappa_n}{\kappa_n + 1} \frac{\alpha_n}{\beta_n}, \\ \nu_\mathrm{p} &= 2 \alpha_n.\end{split}\]

class bayesml.normal.GenModel(mu=0.0, tau=1.0, h_m=0.0, h_kappa=1.0, h_alpha=1.0, h_beta=1.0, seed=None)#

Bases: Generative

The stochastic data generative model and the prior distribution

Parameters:

mufloat, optional: a real number, by default 0.0
taufloat, optional: a positive real number, by default 1.0
h_mfloat, optional: a real number, by default 0.0
h_kappafloat, optional: a positive real number, by default 1.0
h_alphafloat, optional: a positive real number, by default 1.0
h_betafloat, optional: a positive real number, by default 1.0
seed{None, int}, optional: A seed to initialize numpy.random.default_rng(), by default None

Methods

`gen_params`()	Generate the parameter from the prior distribution.
`gen_sample`(sample_size)	Generate a sample from the stochastic data generative model.
`get_constants`()	Get constants of GenModel.
`get_h_params`()	Get the hyperparameters of the prior distribution.
`get_params`()	Get the parameter of the sthocastic data generative model.
`load_h_params`(filename)	Load the hyperparameters to h_params.
`load_params`(filename)	Load the parameters saved by `save_params`.
`save_h_params`(filename)	Save the hyperparameters using python `pickle` module.
`save_params`(filename)	Save the parameters using python `pickle` module.
`save_sample`(filename, sample_size)	Save the generated sample as NumPy `.npz` format.
`set_h_params`([h_m, h_kappa, h_alpha, h_beta])	Set the hyperparameters of the prior distribution.
`set_params`([mu, tau])	Set the parameter of the sthocastic data generative model.
`visualize_model`([sample_size, hist_bins])	Visualize the stochastic data generative model and generated samples.

get_constants()#

Get constants of GenModel.

This model does not have any constants. Therefore, this function returns an emtpy dict {}.

Returns:

constantsan empty dict

set_h_params(h_m=None, h_kappa=None, h_alpha=None, h_beta=None)#

Set the hyperparameters of the prior distribution.

Parameters:

h_mfloat, optional: a real number, by default None
h_kappafloat, optional: a positive real number, by default None
h_alphafloat, optional: a positive real number, by default None
h_betafloat, optional: a positive real number, by default None

get_h_params()#

Get the hyperparameters of the prior distribution.

Returns:

h_paramsdict of {str: float}

"h_m" : The value of self.h_m
"h_kappa" : The value of self.h_kappa
"h_alpha" : The value of self.h_alpha
"h_beta" : The value of self.h_beta

gen_params()#

Generate the parameter from the prior distribution.

The generated vaule is set at self.mu and self.tau.

set_params(mu=None, tau=None)#

Set the parameter of the sthocastic data generative model.

Parameters:

mufloat, optional: a real number \(mu \in \mathbb{R}\), by default None.
taufloat, optional: a positive real number, by default None.

get_params()#

Get the parameter of the sthocastic data generative model.

Returns:

paramsdict of {str:float}

"mu" : The value of self.mu
"tau" : The value of self.tau

gen_sample(sample_size)#

Generate a sample from the stochastic data generative model.

Parameters:

sample_sizeint: A positive integer

Returns:

xnumpy ndarray: 1 dimensional array whose size is sammple_size.

save_sample(filename, sample_size)#

Save the generated sample as NumPy .npz format.

It is saved as a NpzFile with keyword: “x”.

Parameters:

filenamestr: The filename to which the sample is saved. .npz will be appended if it isn’t there.
sample_sizeint: A positive integer

See also

numpy.savez_compressed

visualize_model(sample_size=1000, hist_bins=10)#

Visualize the stochastic data generative model and generated samples.

Parameters:

sample_sizeint, optional: A positive integer, by default 1000
hist_binsfloat, optional: A positive float, by default 10

Examples

>>> from bayesml import normal
>>> model = normal.GenModel()
>>> model.visualize_model()

class bayesml.normal.LearnModel(h0_m=0.0, h0_kappa=1.0, h0_alpha=1.0, h0_beta=1.0)#

Bases: Posterior, PredictiveMixin

The posterior distribution and the predictive distribution.

Parameters:

h0_mfloat, optional: a real number, by default 0.0
h0_kappafloat, optional: a positive real number, by default 1.0
h0_alphafloat, optional: a positive real number, by default 1.0
h0_betafloat, optional: a positive real number, by default 1.0

Attributes:

hn_mfloat: a real number
hn_kappafloat: a positive real number
hn_alphafloat: a positive real number
hn_betafloat: a positive real number
p_mufloat: a real number
p_lambdafloat: a positive real number
p_nufloat: a positive real number

Methods

`calc_log_marginal_likelihood`()	Calculate log marginal likelihood
`calc_pred_dist`()	Calculate the parameters of the predictive distribution.
`estimate_interval`([credibility])	Credible interval of the parameter.
`estimate_params`([loss, dict_out])	Estimate the parameter of the stochastic data generative model under the given criterion.
`get_constants`()	Get constants of LearnModel.
`get_h0_params`()	Get the initial values of the hyperparameters of the posterior distribution.
`get_hn_params`()	Get the hyperparameters of the posterior distribution.
`get_p_params`()	Get the parameters of the predictive distribution.
`load_h0_params`(filename)	Load the hyperparameters to h0_params.
`load_hn_params`(filename)	Load the hyperparameters to hn_params.
`make_prediction`([loss])	Predict a new data point under the given criterion.
`overwrite_h0_params`()	Overwrite the initial values of the hyperparameters of the posterior distribution by the learned values.
`pred_and_update`(x[, loss])	Predict a new data point and update the posterior sequentially.
`reset_hn_params`()	Reset the hyperparameters of the posterior distribution to their initial values.
`save_h0_params`(filename)	Save the hyperparameters using python `pickle` module.
`save_hn_params`(filename)	Save the hyperparameters using python `pickle` module.
`set_h0_params`([h0_m, h0_kappa, h0_alpha, ...])	Set the hyperparameters of the prior distribution.
`set_hn_params`([hn_m, hn_kappa, hn_alpha, ...])	Set updated values of the hyperparameter of the posterior distribution.
`update_posterior`(x)	Update the hyperparameters of the posterior distribution using traning data.
`visualize_posterior`()	Visualize the posterior distribution for the parameter.

get_constants()#

Get constants of LearnModel.

This model does not have any constants. Therefore, this function returns an emtpy dict {}.

Returns:

constantsan empty dict

set_h0_params(h0_m=None, h0_kappa=None, h0_alpha=None, h0_beta=None)#

Set the hyperparameters of the prior distribution.

Parameters:

h0_mfloat, optional: a real number, by default None
h0_kappafloat, optional: a positive real number, by default None
h0_alphafloat, optional: a positive real number, by default None
h0_betafloat, optional: a positive real number, by default None

get_h0_params()#

Get the initial values of the hyperparameters of the posterior distribution.

Returns:

h0_paramsdict of {str: float}

"h0_m" : The value of self.h0_m
"h0_kappa" : The value of self.h0_kappa
"h0_alpha" : The value of self.h0_alpha
"h0_beta" : The value of self.h0_beta

set_hn_params(hn_m=None, hn_kappa=None, hn_alpha=None, hn_beta=None)#

Set updated values of the hyperparameter of the posterior distribution.

Parameters:

hn_mfloat, optional: a real number, by default None
hn_kappafloat, optional: a positive real number, by default None
hn_alphafloat, optional: a positive real number, by default None
hn_betafloat, optional: a positive real number, by default None

get_hn_params()#

Get the hyperparameters of the posterior distribution.

Returns:

hn_paramsdict of {str: float}

"hn_m" : The value of self.hn_m
"hn_kappa" : The value of self.hn_kappa
"hn_alpha" : The value of self.hn_alpha
"hn_beta" : The value of self.hn_beta

update_posterior(x)#

Update the hyperparameters of the posterior distribution using traning data.

Parameters:

xnumpy.ndarray: Real numbers.

estimate_params(loss='squared', dict_out=False)#

Estimate the parameter of the stochastic data generative model under the given criterion.

Note that the criterion is applied to estimating mu and tau independently. Therefore, a tuple of the student’s t-distribution and the gamma distribution will be returned when loss=”KL”

Parameters:

lossstr, optional: Loss function underlying the Bayes risk function, by default “squared”. This function supports “squared”, “0-1”, “abs”, and “KL”.
dict_outbool, optional: If True, output will be a dict, by default False.

Returns:

estimatestuple of {float, None, or rv_frozen}

mu_hat : the estimate for mu
tau_hat : the estimate for tau

The estimated values under the given loss function. If it is not exist, None will be returned. If the loss function is “KL”, the posterior distribution itself will be returned as rv_frozen object of scipy.stats.

bayesml.normal package

Contents

bayesml.normal package#

Module contents#