Research Article: Mix and match. A simulation study on the impact of mixed-treatment comparison methods on health-economic outcomes

Date Published: February 2, 2017

Publisher: Public Library of Science

Author(s): Pepijn Vemer, Maiwenn J. Al, Mark Oppe, Maureen P. M. H. Rutten-van Mölken, Iratxe Puebla.


Decision-analytic cost-effectiveness (CE) models combine many parameters, often obtained after meta-analysis.

We compared different methods of mixed-treatment comparison (MTC) to combine transition and event probabilities derived from several trials, especially with respect to health-economic (HE) outcomes like (quality adjusted) life years and costs.

Trials were drawn from a simulated reference population, comparing two of four fictitious interventions. The goal was to estimate the CE between two of these. The amount of heterogeneity between trials was varied in scenarios. Parameter estimates were combined using direct comparison, MTC methods proposed by Song and Puhan, and Bayesian generalized linear fixed effects (GLMFE) and random effects models (GLMRE). Parameters were entered into a Markov model. Parameters and HE outcomes were compared with the reference population using coverage, statistical power, bias and mean absolute deviation (MAD) as performance indicators. Each analytical step was repeated 1,000 times.

The direct comparison was outperformed by the MTC methods on all indicators, Song’s method yielded low bias and MAD, but uncertainty was overestimated. Puhan’s method had low bias and MAD and did not overestimate uncertainty. GLMFE generally had the lowest bias and MAD, regardless of the amount of heterogeneity, but uncertainty was overestimated. GLMRE showed large bias and MAD and overestimated uncertainty. Song’s and Puhan’s methods lead to the least amount of uncertainty, reflected in the shape of the CE acceptability curve. GLMFE showed slightly more uncertainty.

Combining direct and indirect evidence is superior to using only direct evidence. Puhan’s method and GLMFE are preferred.

Partial Text

In 2006, The Netherlands implemented conditional reimbursement of potentially innovative, but expensive hospital drugs, on the condition that further real-life evidence is collected.[1] After four years, a new reimbursement decision is made, based on all evidence available. Unfortunately, new drugs are often compared to placebo or standard care and the interventions of interest vary by country or over time. Trials incorporating all competing interventions are impractical at best, impossible at worst.[2] This means that a direct, head-to-head comparison may not be available. If a comparison via a common comparator is available, an indirect treatment comparison (ITC) can be used to combine the relative effects of the two treatments versus the a common comparator.[3] With three or more interventions, there may be direct evidence for some pairs of interventions, while other pairs can be compared only via one or more of the other interventions. Techniques to analyze all the available evidence simultaneously are called mixed treatment comparisons (MTC).

In this study, we compared four methods of indirect meta-analysis in a simulation study and judged their statistical performance by creating a gold standard. On a parameter level, Puhan’s method (PUHAN) showed the best performance, overestimating uncertainty for the fewest parameters with low bias and MAD. Song’s method (SONG) and the Bayesian fixed effect generalized linear model (GLMFE) also had generally low bias and MAD.




0 0 vote
Article Rating
Notify of
Inline Feedbacks
View all comments