Model Diagnostic

The BI class can compute model diagnostics for a given model.

Lets consider the following model for a linear regression:

Y_i \sim \text{Normal}(\alpha + \beta X_i, \sigma)

\alpha \sim \text{Normal}(0, 1)

\beta \sim \text{Normal}(0, 1)

\sigma \sim \text{Uniform}(0, 50)

Code

from BI import bi
import jax.numpy as jnp

# setup platform------------------------------------------------
m = bi(platform="cpu")

# import data ------------------------------------------------
data_path = m.load.howell1(only_path=True)
m.data(data_path, sep=";")
m.df = m.df[m.df.age > 18]
m.scale(data=["weight"])


# define model ------------------------------------------------
def model(weight, height):
    a = m.dist.normal(178, 20, name="a")
    b = m.dist.log_normal(0, 1, name="b")
    s = m.dist.uniform(0, 50, name="s")
    m.dist.normal(a + b * weight, s, obs=height, shape=(weight.shape[0],))


# Run sampler ------------------------------------------------
m.fit(model, num_samples=500, num_chains=4)
m.summary()

/home/sosa/work/3.12venv/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning:

IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html

BI v 0.0.45 package loaded
jax.local_device_count 32

  0%|          | 0/1500 [00:00<?, ?it/s]Compiling.. :   0%|          | 0/1500 [00:00<?, ?it/s]
  0%|          | 0/1500 [00:00<?, ?it/s]
Compiling.. :   0%|          | 0/1500 [00:00<?, ?it/s]

  0%|          | 0/1500 [00:00<?, ?it/s]

Compiling.. :   0%|          | 0/1500 [00:00<?, ?it/s]


  0%|          | 0/1500 [00:00<?, ?it/s]


Compiling.. :   0%|          | 0/1500 [00:00<?, ?it/s]Running chain 0:   0%|          | 0/1500 [00:00<?, ?it/s]
Running chain 1:   0%|          | 0/1500 [00:00<?, ?it/s]

Running chain 2:   0%|          | 0/1500 [00:00<?, ?it/s]


Running chain 3:   0%|          | 0/1500 [00:00<?, ?it/s]Running chain 2: 100%|██████████| 1500/1500 [00:00<00:00, 1805.79it/s]
Running chain 0: 100%|██████████| 1500/1500 [00:00<00:00, 1793.27it/s]
Running chain 1: 100%|██████████| 1500/1500 [00:00<00:00, 1794.71it/s]
Running chain 3: 100%|██████████| 1500/1500 [00:00<00:00, 1795.86it/s]

	mean	sd	hdi_5.5%	hdi_94.5%	mcse_mean	mcse_sd	ess_bulk	ess_tail	r_hat
a	154.65	0.27	154.25	155.12	0.01	0.01	1873.35	1329.85	1.0
b	5.81	0.28	5.39	6.28	0.01	0.01	1849.54	1519.87	1.0
s	5.14	0.20	4.83	5.48	0.00	0.00	1982.99	1415.96	1.0

List of all available diagnostics

For additional documentation check the diagnostics API reference

Predictions from model based on specific data value

Code

m.sample() # Predictions from model base on data in data_on_model
m.sample(data=dict(weight=jnp.array([0.4])), remove_obs=False)# Predictions from a given value

/home/sosa/work/BI/BI/Main/main.py:590: UserWarning:

Sample's batch dimension size 2000 is different from the provided 1 num_samples argument. Defaulting to 2000.

{'x': Array([[159.21693302],
        [163.9983087 ],
        [157.6438639 ],
        ...,
        [162.61740508],
        [148.05320942],
        [161.35796436]], dtype=float64)}

Forest plot of estimated values

Code

m.diag.forest()

Density plots of the posterior distribution

Code

m.diag.density()

Posterior distribution plots

Code

m.diag.posterior()

Trace plots for MCMC chains

Code

m.diag.plot_trace()

Pairwise plots of the posterior distribution

Code

m.diag.pair()

Plot autocorrelation of MCMC chains

Code

m.diag.autocor()

Create rank plots for MCMC chains

Code

m.diag.rank()

Evolution of effective sample size across iterations

Code

m.diag.plot_ess()

Pareto-smoothed

Code

m.diag.loo()

Computed from 2000 posterior samples and 346 observations log-likelihood matrix.

         Estimate       SE
elpd_loo -1058.44    14.67
p_loo        3.14        -
------

Pareto k diagnostic values:
                         Count   Pct.
(-Inf, 0.70]   (good)      346  100.0%
   (0.70, 1]   (bad)         0    0.0%
   (1, Inf)   (very bad)    0    0.0%

Widely applicable information criterion

Code

m.diag.WAIC()

Computed from 2000 posterior samples and 346 observations log-likelihood matrix.

          Estimate       SE
elpd_waic -1058.43    14.67
p_waic        3.13        -