Select Page

Bootstrap

[Bootstrap is not available in versions prior to 2024R1]

Introduction

The Bootstrap module in Monolix provides a robust method to assess parameter uncertainty, offering an alternative to calculating standard errors via inversion of the Fisher Information Matrix. This approach becomes particularly valuable when facing issues such as NaNs in standard errors due to numerical errors in matrix inversion or biases in results caused by incorrect assumptions of asymptotic normality for parameter estimates. Bootstrap overcomes these challenges by sampling many replicate datasets and re-estimating parameters on each replicate.

While powerful, bootstrap comes with certain drawbacks:

  • Running many replicates for population parameter estimation can be time-consuming. Bootstrap in Monolix can be used with distributed calculation (documentation page under construction).
  • Saving a large number of new datasets and results may raise storage issues. You can choose in the settings whether to save sampled datasets and results.

 

Accessing the bootstrap module

Bootstrap can be accessed under the “Perspective” menu or with a shortcut next to “Run” in the “Statistical model & Tasks” tab:

   

Available bootstrap settings

Users can customize the following settings:

  • Number of runs: number of bootstrap replicates
  • Estimation tasks: Tasks to run in addition to SAEM: Standard Errors, Log-Likelihood
  • Initial values: Whether bootstrap runs should start their estimation from the same initial values as the initial run, or from the final estimated values from the initial run.
  • Sampling method: Type of bootstrap:
    • Nonparametric Bootstrap: new datasets are sampled from the initial dataset for each replicate.
    • Parametric Bootstrap: new datasets are simulated from the model for each replicate. If the initial dataset has censored data, censoring limits to apply to the simulated datasets can be specified to avoid bias.
  • Sample size: By default it is set equal to the size of the original dataset but can be adjusted for computational efficiency.
  • Stratified resampling: Original distribution preservation for categorical covariates
  • Confidence interval level: for uncertainty on parameters.
  • Option to save or not save bootstrap datasets and results.
  • Option to replace or not bootstrap replicates that have a failed convergence.

 

Running bootstrap

After clicking on “Run”, bootstrap is launched and the population parameters estimates from all runs already done are shown as a table and as a plot of their medians and their confidence intervals with respect to the bootstrap iterations:

If you have stopped bootstrap before the last run, or you want to add more runs to your bootstrap results, you can resume bootstrap so that the runs already done will be reused instead of being rerun:

Results

Bootstrap results reported in the interface and the output files include:

  • All bootstrap estimates displayed in tables and as distribution plots,

   

  • Table of statistics on bootstrap estimates including confidence intervals and standard deviations, and comparison of bootstrap means with reference estimated values,

 

  • The possibility to load each bootstrap run if it has been saved:

In the case of non-parametric bootstrap, resampled datasets used in bootstrap runs have new subject identifiers defined as integers in the column tagged as ID, and include an additional column named “original_id” with the corresponding subject identifiers from the initial dataset.