Date Published: December 11, 2018
Publisher: Public Library of Science
Author(s): Rowan Iskandar, Vygintas Gontis.
Following its introduction over three decades ago, the cohort model has been used extensively to model population trajectories over time in decision-analytic modeling studies. However, the stochastic process underlying cohort models has not been properly described. In this study, we explicate the stochastic process underlying a cohort model, by carefully formulating the dynamics of populations across health states and assigning probability rules on these dynamics. From this formulation, we explicate a mathematical representation of the system, which is given by the master equation. We solve the master equation by using the probability generation function method to obtain the explicit form of the probability of observing a particular realization of the system at an arbitrary time. The resulting generating function is used to derive the analytical expressions for calculating the mean and the variance of the process. Secondly, we represent the cohort model by a difference equation for the number of individuals across all states. From the difference equation, a continuous-time cohort model is recovered and takes the form of an ordinary differential equation. To show the equivalence between the derived stochastic process and the cohort model, we conduct a numerical exercise. We demonstrate that the population trajectories generated from the formulas match those from the cohort model simulation. In summary, the commonly-used cohort model represent the average of a continuous-time stochastic process on a multidimensional integer lattice governed by a master equation. Knowledge of the stochastic process underlying a cohort model provides a theoretical foundation for the modeling method.
Decision models have been used in various applications from clinical decision making to screening guideline development. The objective of decision modeling is to integrate and present evidence within a coherent and explicit mathematical structure that can be used to link evidence to decision-relevant outcomes. In decision modeling, a state-transition Markov model is often used to simulate the prognosis of a patient or a group of patients following an intervention. Beck and Pauker  introduced the modeling method over 30 years ago with the aim to provide a simple tool for prognostic modeling and for practical use in medical decision making. They applied standard methods from Markov chain theory [2, 3] to simulate a life history of an individual which is structured as transitions across various heath states over time, i.e. a stochastic process on a finite state space (S). However, the current literature in decision modeling lacks clarity in the following two concepts. First, a cohort model, an extension of the state-transition model from one individual to a group of individuals, is often used to introduce and describe a stochastic process on S. According to  and the subsequent published tutorials and textbook [4–7], given a matrix of transition probabilities and an initial distribution of counts of individuals across health states, a cohort model generates the life trajectory of a cohort of identical individuals by repeated multiplications of the vector of population counts by the transition probability matrix. This matrix operation alludes to a stochastic process on a much larger state space, i.e. N|S| (compared to S), where N and |S| denote the set of natural numbers and the number of states, respectively. For example, S may be a partitioning on an individual’s health state into healthy, sick, or dead. In contrast, N|S| may refer to a partitioning of a cohort of individuals into the numbers of healthy, sick and, dead individuals. The convolution of these two different processes in decision modeling literature lead to the second issue: practitioners are taught that cohort models capture the average behavior of the individuals. [7, 8] However, this claim is often stated without any clear reference to which stochastic process. Frederix et. al.  proposes an ordinary differential equation (ODE)-based method to approximate Markov models whilst acknowledging that the ODEs can describe only “the typical (mean) change,” albeit, without providing any rational or reference. One of the standard textbooks  describes cohort model as a representation of the average experience of patients in the cohort without providing a rationale to support this description. Furthermore, wide acceptance of methodologies does not automatically imply veracity.  In the context of cohort models, the wide acceptance is ingrained by the ISPOR-SMDM Modeling Good Research Practices Task Force-3.  The published best practices cites the work of Beck et al. [1, 4] as the main and only references for cohort models, thereby standardizing cohort models as the recommended method for simulating population trajectories despite of the lack of a proper description of the theoretical support in the decision modeling area.
In this study, we explicate the stochastic process underlying the commonly used cohort model, defined as a continuous-time discrete-state stochastic process, in the context where population counts are of interest. First, We start the derivation by specifying a stochastic state-transition model for an individual following Beck and Pauker: a stochastic process on state space S.  We then extend the model to a cohort of individuals by formulating a set of postulates for the cohort dynamics: a stochastic process on N|S|. The evolution equation representing the stochastic process for the cohort dynamics in the form of master equation is then obtained. We then solve the master equation to get the probability function of individuals across all states, which completely characterizes stochastic nature of the dynamics, by using the generating function method. Based on the explicit form of the generating function, the analytical formulas for the average and variance are obtained. This measure of variance quantifies the inherent variability of the population trajectories (aleatory uncertainty  or first-order uncertainty ) and may be useful for quantifying the extent of variability in the mean prediction. In our simulation example, we show that to estimate the variance of the population counts across states, we need to replicate the microsimulation population a number of times in addition to the individual Monte Carlo runs within each population, which is a potentially computationally intensive task and is similar to applying uncertainty analysis to a microsimulation model.  In particular, if some of the events in the model are rare and/or the expected differences in the effect sizes among strategies are small, we expect the required number of Monte Carlo runs to be high. Our theoretical result provides a more direct and non-computationally demanding approach through the explicit formula for the variance. The derived probability function, when all individuals start in one state (a typical scenario in most applications) takes the form of a multinomial distribution and is a known result from the stochastic compartmental model literature . In applications where the relevant population comprises of an actual cohort (e.g. birth cohorts), the individuals are often, prior to an implementation of an intervention, distributed across health states. For this purpose, we also derive the probability function and formulas for the moments, for an arbitrary initial condition. Lastly, we show the equivalence between the average of the derived stochastic process and the cohort model; thereby substantiating the prevailing notion of cohorts models as average process.
Many decisions in health are often made with incomplete understanding of the underlying phenomena. In such cases, stochastic models can serve as powerful tools for aiding decision-makers in synthesizing evidence from different sources, estimating decision-relevant outcomes, and quantifying the impact of uncertainty on decision-making. In particular, the cohort model provides an easy-to-implement method for modeling recurrent events over time and has been used extensively in many applications ranging from clinical decision making to policy evaluations. A cohort model represents the average of a stochastic process which is presented by a master equation. The derived stochastic process rectifies and unifies the prevailing descriptions of cohort models in the literature and practice. Knowledge of the underlying theoretical framework would reaffirm the prevailing beliefs in the veracity and validity of the method and thereby increase confidence in conclusions derived from studies using cohort models. Future studies should focus on increasing the flexibility of cohort models and decreasing computational time by leveraging the relationship between the derived process and other stochastic processes.