Select Page

Automatic covariate model building

Starting from the 2019 version, automatic covariate model building algorithms are implemented in Monolix, and can be chosen in the “Initialization” tab of the Model building panel:

 

SCM

This is the classic stepwise covariate modeling method. A set of iterations of forward selection is followed by a set of iterations of backward selection.

In the forward selection, at each step, each of the remaining (i.e not yet included) parameter-covariate relationships are added to the model in an univariate model (one model per relationship), and run. Among all models, the model that improves some criteria (LRT, or BICc) most is selected and taken forward to the next step. During backward elimination, parameter-covariate relationships are removed in an univariate manner.

This method is effective but expensive in terms of number of runs, because it requires to estimate the population parameters with SAEM and simulate individual parameters from Conditional distributions for each evaluated model.

 

COSSAC

COSSAC (COnditional Sampling use for Stepwise Approach based on Correlation tests) makes use of the information contained in the base model run to choose which covariate to try first (instead of trying all covariates “blindly” as in SCM).

Indeed, the correlation between the individual parameters (or random effects) and the covariates hints at possibly relevant parameter-covariate relationships. If the EBEs (empirical Bayes estimates) are used, shrinkage may bias the result. COSSAC instead uses samples from the a posteriori conditional distribution (available as “conditional distribution” task in Monolix) to calculate the correlation between the random effects and covariates. A p-value can be derived using the Pearson’s correlation test for continuous covariate and ANOVA for categorical covariate. The p-values are used to sort all the random effect-covariate relationships, whether they are included in the model or not.

The iterations of COSSAC alternate between a forward selection and a backward selection, depending on the results of the correlation tests:

  • Initialization:
    • Run the base model (population parameter estimation, conditional distribution sampling, and log-likelihood estimation).
    • Calculate the p-values of all the parameter-covariate relationships using Pearson’s correlation tests and ANOVA (automatically done in Monolix)
    • Start with a backward selection.
  • Forward selection:
    1. Add the covariate with the smallest correlation p-value (among the remaining parameter-covariate relationships) to the model, or the next smallest if the smallest has already been tried, and so until no correlation p-values above a threshold remain
    2. Run the model with the same initial values as the base model (initial estimates for the new beta parameters are 0)
    3. Accept/reject the relationship based on the likelihood ratio test or BIC: the model is not retained if the criteria does not improve (with a threshold for the likelihood ratio test)
    4. Calculate all parameter-covariate correlation p-values
    5. Try a backward selection
  • Backward selection:
    1. Among the covariates present in the model, remove the covariate with the highest (less significant) correlation p-value, or the next highest if the highest has already been tried, and so until no correlation p-values below a threshold remain
    2. Run the model with the same initial values as the base model (initial estimates for the new beta parameters are 0)
    3. Accept/reject the relationship removal based on the likelihood ratio test or BIC: the model is not retained if the criteria does not improve (with a threshold for the likelihood ratio test)
    4. Calculate all parameter-covariate relationship correlation p-value
    5. Try a forward selection
  • The algorithm continues until no forward or backward selection is possible, or after testing 10 new relationships on the same model.
  • Relationships between covariates and parameters without variability can not be explored with COSSAC, therefore parameters without variability are handled with SCM at the end of COSSAC.

 

Example

The major difference between SCM and COSSAC is that very few iterations are usually needed with COSSAC before deciding to add or remove a parameter-covariate relationship: between 1 and 10, depending on the number of significant correlations and the improvement brought by each new relationship.

The schema below illustrates this difference on an example with 4 parameters (ka, V, Cl, Tlag) and 3 covariates (Age, Sex, WT). Purple arrows represent forward selections while orange arrows represent backward selections. Starting from a model with no relationship between parameters and covariates, each model run in a new iteration is represented with a rectangle, colored in blue if the model is kept as a starting point for the next iteration, or in grey if the model does not bring improvement. Final models are colored in green. In the case of COSSAC, 1 iteration is sufficient to update the model except at the second step, because the first relationship tried (Sex on Tlag) does not improve the criteria. At the last step only 3 relationships with a remaining p-value below the threshold have to be tested before finishing. In the case of SCM, all models run are not displayed but the number of iterations at each step is written. In total, 46 models are run during the SCM procedure, while COSSAC needs only 8 models.

 

covSAMBA – COSSAC

With “covSAMBA – COSSAC”, it is possible to run COSSAC after just one iteration of SAMBA applied to the covariate effects.

Since an iteration of SAMBA, like the Proposal, evaluates a set of covariate models without estimating their parameters with SAEM, it is very fast. This iteration can provide a starting model that is much closer to the final covariate model than the initial model. COSSAC is then used to find more precisely the best covariate model.

 

Settings

The linearization method is selected by default to compute the log-likelihood of the estimated model at each iteration. It can be unselected to use importance sampling.

The improvement can be evaluated with two different criteria based on the log-likelihood, that can be selected in the settings (available via the icon next to Run):

  • BICc (by default)
  • LRT (likelihood ratio threshold): by default the forward threshold is 0.1 and the backward threshold is 0.05. These values can be changed in the settings.

By default, the correlation thresholds to add new covariate effects in COSSAC (forward) is 0.5 and the correlation threshold to remove some covariate effects in COSSACc(backward) is 0.01. These values can be changed in the settings.

 

Selecting covariates and parameters

It is possible to select part of the covariates and individual parameters to be used in the algorithm:

Moreover, a panel “Locked relationships” can be opened to lock in or lock out some covariate-parameter relationships among the ones that are available:

 

Results

The results of the model building are displayed in a tab Results, with the list of model run at each iteration (see for example the figure below). By default they are displayed by order of iteration, except the best model which is displayed in first position, highlighted in green. Differences with the previous iteration are highlighted in yellow. Parameters or covariates that have been excluded from the covariate search appear in grey. Locked in or locked out relationships are also indicated with a blue or red icon.

Note that the table of iterations can be sorted by iteration number or criteria (see green marks below). Buttons “export and load” (see blue mark below) can also be used to export the model estimated at this iteration as a new Monolix project with a new name.