bayesml.categorical package

bayesml.categorical package#

Module contents#

The categorical distribution with the dirichlet prior distribution

The stochastic data generative model is as follows:

\(d\in \mathbb{Z}\): a dimension (\(d \geq 2\))
\(\boldsymbol{x} \in \{ 0, 1\}^d\): a data point, (a one-hot vector, i.e., \(\sum_{k=1}^d x_k=1\))
\(\boldsymbol{\theta} \in [0, 1]^d\): a parameter, (\(\sum_{k=1}^d \theta_k=1\))

\[p(\boldsymbol{x} | \boldsymbol{\theta}) = \mathrm{Cat}(\boldsymbol{x}|\boldsymbol{\theta}) = \prod_{k=1}^d \theta_k^{x_k},\]

\[\begin{split}\mathbb{E}[\boldsymbol{x} | \boldsymbol{\theta}] &= \boldsymbol{\theta}, \\ \mathbb{V}[x_k | \boldsymbol{\theta}] &= \theta_k (1 - \theta_k), \\ \mathrm{Cov}[x_k, x_{k'} | \boldsymbol{\theta}] &= -\theta_k \theta_{k'}.\end{split}\]

The prior distribution is as follows:

\(\boldsymbol{\alpha}_0 \in \mathbb{R}_{>0}^d\): a hyperparameter
\(\Gamma (\cdot)\): the gamma function
\(\tilde{\alpha}_0 = \sum_{k=1}^d \alpha_{0,k}\)
\(C(\boldsymbol{\alpha}_0)=\frac{\Gamma(\tilde{\alpha}_0)}{\Gamma(\alpha_{0,1})\cdots\Gamma(\alpha_{0,d})}\)

\[p(\boldsymbol{\theta}) = \mathrm{Dir}(\boldsymbol{\theta}|\boldsymbol{\alpha}_0) = C(\boldsymbol{\alpha}_0)\prod_{k=1}^d\theta_k^{\alpha_{0,k}-1},\]

\[\begin{split}\mathbb{E}[\boldsymbol{\theta}] &= \frac{\boldsymbol{\alpha}_0}{\tilde{\alpha}_0}, \\ \mathbb{V}[\theta_k] &= \frac{1}{\tilde{\alpha}_0 + 1} \frac{\alpha_{0,k}}{\tilde{\alpha}_0} \left(1 - \frac{\alpha_{0,k}}{\tilde{\alpha}_0} \right), \\ \mathrm{Cov}[\theta_k, \theta_{k'}] &= - \frac{1}{\tilde{\alpha}_0 + 1} \frac{\alpha_{0,k}}{\tilde{\alpha}_0} \frac{\alpha_{0,k'}}{\tilde{\alpha}_0}.\end{split}\]

The posterior distribution is as follows:

\(\boldsymbol{x}^n = (\boldsymbol{x}_1, \boldsymbol{x}_2, \dots , \boldsymbol{x}_n) \in \{ 0, 1\}^{d\times n}\): given data
\(\boldsymbol{\alpha}_n \in \mathbb{R}_{>0}^d\): a hyperparameter
\(\tilde{\alpha}_n = \sum_{k=1}^d \alpha_{n,k}\)
\(C(\boldsymbol{\alpha}_n)=\frac{\Gamma(\tilde{\alpha}_n)}{\Gamma(\alpha_{n,1})\cdots\Gamma(\alpha_{n,d})}\)

\[p(\boldsymbol{\theta} | \boldsymbol{x}^n) = \mathrm{Dir}(\boldsymbol{\theta}|\boldsymbol{\alpha}_n) = C(\boldsymbol{\alpha}_n)\prod_{k=1}^d\theta_k^{\alpha_{n,k}-1},\]

\[\begin{split}\mathbb{E}[\boldsymbol{\theta} | \boldsymbol{x}^n] &= \frac{\boldsymbol{\alpha}_n}{\tilde{\alpha}_n}, \\ \mathbb{V}[\theta_k | \boldsymbol{x}^n] &= \frac{1}{\tilde{\alpha}_n + 1} \frac{\alpha_{n,k}}{\tilde{\alpha}_n} \left(1 - \frac{\alpha_{n,k}}{\tilde{\alpha}_n} \right), \\ \mathrm{Cov}[\theta_k, \theta_{k'} | \boldsymbol{x}^n] &= - \frac{1}{\tilde{\alpha}_n + 1} \frac{\alpha_{n,k}}{\tilde{\alpha}_n} \frac{\alpha_{n,k'}}{\tilde{\alpha}_n},\end{split}\]

where the updating rule of the hyperparameters is as follows.

\[\alpha_{n,k} = \alpha_{0,k} + \sum_{i=1}^n x_{i,k}, \quad (k \in \{ 1, 2, \dots , d \}).\]

The predictive distribution is as follows:

\(\boldsymbol{x}_{n+1} \in \{ 0, 1\}^d\): a new data point
\(\boldsymbol{\theta}_\mathrm{p} \in [0, 1]^d\): the hyperparameter of the posterior (\(\sum_{k=1}^d \theta_{\mathrm{p},k} = 1\))

\[p(\boldsymbol{x}_{n+1} | \boldsymbol{x}^n) = \mathrm{Cat}(\boldsymbol{x}_{n+1}|\boldsymbol{\theta}_\mathrm{p}) = \prod_{k=1}^d \theta_{\mathrm{p},k}^{x_{n+1,k}},\]

\[\begin{split}\mathbb{E}[\boldsymbol{x}_{n+1} | \boldsymbol{x}^n] &= \boldsymbol{\theta}_\mathrm{p}, \\ \mathbb{V}[x_{n+1,k} | \boldsymbol{x}^n] &= \theta_{\mathrm{p},k} (1 - \theta_{\mathrm{p},k}), \\ \mathrm{Cov}[x_{n+1,k}, x_{n+1,k'} | \boldsymbol{x}^n] &= -\theta_{\mathrm{p},k} \theta_{\mathrm{p},k'},\end{split}\]

where the parameters are obtained from the hyperparameters of the posterior distribution as follows:

\[\theta_{\mathrm{p},k} = \frac{\alpha_{n,k}}{\sum_{k=1}^d \alpha_{n,k}}, \quad (k \in \{ 1, 2, \dots , d \}).\]

class bayesml.categorical.GenModel(c_degree, theta_vec=None, h_alpha_vec=None, seed=None)#

Bases: Generative

The stochastic data generative model and the prior distribution

Parameters:

c_degreeint: a positive integer.
theta_vecnumpy ndarray, optional: a real vector in \([0, 1]^d\), by default [1/d, 1/d, … , 1/d]
h_alpha_vecnumpy ndarray, optional: a vector of positive real numbers, by default [1/2, 1/2, … , 1/2]. If a single real number is input, it will be broadcasted.
seed{None, int}, optional: A seed to initialize numpy.random.default_rng(), by default None

Methods

`gen_params`()	Generate the parameter from the prior distribution.
`gen_sample`(sample_size[, onehot])	Generate a sample from the stochastic data generative model.
`get_constants`()	Get constants of GenModel.
`get_h_params`()	Get the hyperparameters of the prior distribution.
`get_params`()	Get the parameter of the sthocastic data generative model.
`load_h_params`(filename)	Load the hyperparameters to h_params.
`load_params`(filename)	Load the parameters saved by `save_params`.
`save_h_params`(filename)	Save the hyperparameters using python `pickle` module.
`save_params`(filename)	Save the parameters using python `pickle` module.
`save_sample`(filename, sample_size[, onehot])	Save the generated sample as NumPy `.npz` format.
`set_h_params`([h_alpha_vec])	Set the hyperparameters of the prior distribution.
`set_params`([theta_vec])	Set the parameter of the sthocastic data generative model.
`visualize_model`([sample_size, sample_num])	Visualize the stochastic data generative model and generated samples.

get_constants()#

Get constants of GenModel.

Returns:

constantsdict of {str: int}

"c_degree" : the value of self.c_degree

set_h_params(h_alpha_vec=None)#

Set the hyperparameters of the prior distribution.

Parameters:

h_alpha_vecnumpy ndarray, optional: a vector of positive real numbers, by default None. If a single real number is input, it will be broadcasted.

get_h_params()#

Get the hyperparameters of the prior distribution.

Returns:

h_params{str:numpy ndarray}: {"h_alpha_vec": self.h_alpha_vec}

gen_params()#

Generate the parameter from the prior distribution.

The generated vaule is set at self.theta_vec.

set_params(theta_vec=None)#

Set the parameter of the sthocastic data generative model.

Parameters:

pnumpy ndarray, optional: a real vector \(p \in [0, 1]^d\), by default None.

get_params()#

Get the parameter of the sthocastic data generative model.

Returns:

params{str:numpy ndarray}: {"theta_vec":self.theta_vec}

gen_sample(sample_size, onehot=True)#

Generate a sample from the stochastic data generative model.

Parameters:

sample_sizeint: A positive integer
onehotbool, optional: If True, a generated sample will be one-hot encoded, by default True.

Returns:

xnumpy ndarray: An non-negative int array. If onehot option is True, its shape will be (sample_size,c_degree) and each row will be a one-hot vector. If onehot option is False, its shape will be (sample_size,) and each element will be smaller than self.c_degree.

save_sample(filename, sample_size, onehot=True)#

Save the generated sample as NumPy .npz format.

It is saved as a NpzFile with keyword: “x”.

Parameters:

filenamestr: The filename to which the sample is saved. .npz will be appended if it isn’t there.
sample_sizeint: A positive integer
onehotbool, optional: If True, a generated sample will be one-hot encoded, by default True.

See also

numpy.savez_compressed

visualize_model(sample_size=20, sample_num=5)#

Visualize the stochastic data generative model and generated samples.

Parameters:

sample_sizeint, optional: A positive integer, by default 20
sample_numint, optional: A positive integer, by default 5

Examples

>>> from bayesml import categorical
>>> model = categorical.GenModel(3)
>>> model.visualize_model()
theta_vec:[0.33333333 0.33333333 0.33333333]

class bayesml.categorical.LearnModel(c_degree, h0_alpha_vec=None)#

Bases: Posterior, PredictiveMixin

The posterior distribution and the predictive distribution.

Parameters:

c_degreeint: a positive integer.
h0_alpha_vecnumpy.ndarray, optional: a vector of positive real numbers, by default [1/2, 1/2, … , 1/2]. If a single real number is input, it will be broadcasted.

Attributes:

hn_alpha_vecnumpy.ndarray: a vector of positive real numbers
p_theta_vecnumpy.ndarray: a real vector in \([0, 1]^d\)

Methods

`calc_log_marginal_likelihood`()	Calculate log marginal likelihood
`calc_pred_dist`()	Calculate the parameters of the predictive distribution.
`estimate_params`([loss, dict_out])	Estimate the parameter of the stochastic data generative model under the given criterion.
`get_constants`()	Get constants of LearnModel.
`get_h0_params`()	Get the initial values of the hyperparameters of the posterior distribution.
`get_hn_params`()	Get the hyperparameters of the posterior distribution.
`get_p_params`()	Get the parameters of the predictive distribution.
`load_h0_params`(filename)	Load the hyperparameters to h0_params.
`load_hn_params`(filename)	Load the hyperparameters to hn_params.
`make_prediction`([loss, onehot])	Predict a new data point under the given criterion.
`overwrite_h0_params`()	Overwrite the initial values of the hyperparameters of the posterior distribution by the learned values.
`pred_and_update`(x[, loss, onehot])	Predict a new data point and update the posterior sequentially.
`reset_hn_params`()	Reset the hyperparameters of the posterior distribution to their initial values.
`save_h0_params`(filename)	Save the hyperparameters using python `pickle` module.
`save_hn_params`(filename)	Save the hyperparameters using python `pickle` module.
`set_h0_params`([h0_alpha_vec])	Set the hyperparameters of the prior distribution.
`set_hn_params`([hn_alpha_vec])	Set updated values of the hyperparameter of the posterior distribution.
`update_posterior`(x[, onehot])	Update the hyperparameters of the posterior distribution using traning data.
`visualize_posterior`()	Visualize the posterior distribution for the parameter.

get_constants()#

Get constants of LearnModel.

Returns:

constantsdict of {str: int}

"c_degree" : the value of self.c_degree

set_h0_params(h0_alpha_vec=None)#

Set the hyperparameters of the prior distribution.

Parameters:

h0_alpha_vecnumpy ndarray, optional: a vector of positive real numbers, by default None. If a single real number is input, it will be broadcasted.

get_h0_params()#

Get the initial values of the hyperparameters of the posterior distribution.

Returns:

h0_paramsdict of {str: float, numpy.ndarray}: "h0_alpha_vec" : The value of self.h0_alpha_vec

set_hn_params(hn_alpha_vec=None)#

Set updated values of the hyperparameter of the posterior distribution.

Parameters:

hn_alpha_vecnumpy ndarray, optional: a vector of positive real numbers, by default None. If a single real number is input, it will be broadcasted.

get_hn_params()#

Get the hyperparameters of the posterior distribution.

Returns:

hn_paramsdict of {str: numpy.ndarray}: "hn_alpha_vec" : The value of self.hn_alpha_vec

update_posterior(x, onehot=True)#

Update the hyperparameters of the posterior distribution using traning data.

Parameters:

xnumpy.ndarray: A non-negative int array. If onehot option is True, its shape must be (sample_size,c_degree) and each row must be a one-hot vector. If onehot option is False, its shape must be (sample_size,) and each element must be smaller than self.c_degree.
onehotbool, optional: If True, the input sample must be one-hot encoded, by default True.

estimate_params(loss='squared', dict_out=False)#

Estimate the parameter of the stochastic data generative model under the given criterion.

Parameters:

lossstr, optional: Loss function underlying the Bayes risk function, by default “squared”. This function supports “squared”, “0-1”, and “KL”.
dict_outbool, optional: If True, output will be a dict, by default False.

Returns:

estimates{numpy ndarray, float, None, or rv_frozen}: The estimated values under the given loss function. If it is not exist, None will be returned. If the loss function is “KL”, the posterior distribution itself will be returned as rv_frozen object of scipy.stats.

bayesml.categorical package

Contents

bayesml.categorical package#

Module contents#