**Date Published:** February 27, 2019

**Publisher:** Springer International Publishing

**Author(s):** Moustafa M. A. Ibrahim, Sebastian Ueckert, Svetlana Freiberga, Maria C. Kjellsson, Mats O. Karlsson.

http://doi.org/10.1208/s12248-019-0305-2

**Abstract**

**Nonlinear mixed effects models are widely used to describe longitudinal data to improve the efficiency of drug development process or increase the understanding of the studied disease. In such settings, the appropriateness of the modeling assumptions is critical in order to draw correct conclusions and must be carefully assessed for any substantial violations. Here, we propose a new method for structure model assessment, based on assessment of bias in conditional weighted residuals (CWRES). We illustrate this method by assessing prediction bias in two integrated models for glucose homeostasis, the integrated glucose-insulin (IGI) model, and the integrated minimal model (IMM). One dataset was simulated from each model then analyzed with the two models. CWRES outputted from each model fitting were modeled to capture systematic trends in CWRES as well as the magnitude of structural model misspecifications in terms of difference in objective function values (ΔOFVBias). The estimates of CWRES bias were used to calculate the corresponding bias in conditional predictions by the inversion of first-order conditional estimation method’s covariance equation. Time, glucose, and insulin concentration predictions were the investigated independent variables. The new method identified correctly the bias in glucose sub-model of the integrated minimal model (IMM), when this bias occurred, and calculated the absolute and proportional magnitude of the resulting bias. CWRES bias versus the independent variables agreed well with the true trends of misspecification. This method is fast easily automated diagnostic tool for model development/evaluation process, and it is already implemented as part of the Perl-speaks-NONMEM software.**

**Partial Text**

Nonlinear mixed effects (NLME) models are currently advocated to maximize the utilization of gained information throughout all the phases of drug development. These models are adopted for reducing sample size, calculating study power, confirming drug effects, selecting doses, and optimizing trial design as well as supporting final/interim analysis decisions (1). In such settings, the appropriateness of the modeling assumptions is critical in order to draw correct conclusions and the assumptions must be carefully assessed for any substantial violations. Usually, modeling assumptions are assessed from the available knowledge on physiological processes that are to be modeled. However, model misspecifications can occur when the incompatibility of a modeling assumption with the underlying system goes undetected/untested, even though the model appears to give an accurate description of the data (2).

When either of the two data sets was analyzed with the IGI model or data simulated by the IMM was analyzed by the IMM, ΔOFVBias was non-significant for both DVs (glucose and insulin) at documentclass[12pt]{minimal}

usepackage{amsmath}

usepackage{wasysym}

usepackage{amsfonts}

usepackage{amssymb}

usepackage{amsbsy}

usepackage{mathrsfs}

usepackage{upgreek}

setlength{oddsidemargin}{-69pt}

begin{document}$$ {mathcal{X}}_{0.05}^2 $$end{document}X0.052(10 degree of freedom) when time was the IDV, and at documentclass[12pt]{minimal}

usepackage{amsmath}

usepackage{wasysym}

usepackage{amsfonts}

usepackage{amssymb}

usepackage{amsbsy}

usepackage{mathrsfs}

usepackage{upgreek}

setlength{oddsidemargin}{-69pt}

begin{document}$$ {mathcal{X}}_{0.05}^2 $$end{document}X0.052(5) when glucose PRED or insulin PRED was the IDV. When data simulated by the IGI was analyzed with the IMM, ΔOFVBias was significant for glucose versus the three IDVs, but not for insulin as shown in Table I.Table ICalculated ΔOFVBias for the Two Dependent Variables Glucose and Insulin for the IGI Model and the IMM Versus the Three Investigated Independent Variables: Time, Glucose PRED, and Insulin PRED. Significant Bias is Indicated in the Table with ItalicsΔOFVBiasGlucoseInsulinSimulationEstimationTimeGlucose PREDInsulin PREDTimeGlucose PREDInsulin PREDIGIIGI13.646.576.265.562.993.77IMM50.1515.3813.8313.074.023.48IMMIGI6.673.661.2310.956.486.06IMM13.146.438.582.923.343.69The first two rows of the table contains ΔOFVBias when simulating with the IGI model and estimating with both the IGI model and the IMM versus time, glucose PRED and insulin PRED, while the second two rows of the table contains ΔOFVBiaswhen simulating with the IMM and estimating with the two models versus the same IDVs

Nonlinear mixed effects modeling requires assumptions for handling different types of data and the different model components: structural, covariate, and stochastic models; since these assumptions are interconnected with each other such that a violation of one may have consequences for the apparent appropriateness of others, it becomes more challenging to correctly address such violation (8). One of the recently developed methods for model evaluation is CWRES post-processing. By parametric modeling of either the mean or the variance of CWRES distribution, it is possible to identify and quantify if a model misspecification is present and whether this model misspecification arises from the structural model or the stochastic model, in a fast and robust way (9). Based on CWRES modeling, we developed a new method to assess structural assumptions as prediction bias in NLME models developed for continuous data. The new method first calculated the bias in the mean of CWRES distribution, then the deviation between conditional predictions of a misspecified structural model, and expected true structural model, relying on the fact that CWRES under the true structure model is normally distributed with mean 0 and variance 1. We successfully applied the new method to two integrated complex models for glucose homeostasis, the IGI model, and the IMM. Both models claimed an underlying physiologically plausible structure, albeit different, to explain glucose-insulin dynamic interaction with the least possible number of estimated parameters, and so hypothetically both models are less prone to prediction bias. Our method correctly spotted the violation of the underlying structural model assumptions with the highest impact on the IMM performance, similar to % known bias and in agreement to previous investigations (22,23).

Source:

http://doi.org/10.1208/s12248-019-0305-2