Research Article: Assessing the inter-observer variability of Computer-Aided Nodule Assessment and Risk Yield (CANARY) to characterize lung adenocarcinomas

Date Published: June 1, 2018

Publisher: Public Library of Science

Author(s): Erica C. Nakajima, Michael P. Frankland, Tucker F. Johnson, Sanja L. Antic, Heidi Chen, Sheau-Chiann Chen, Ronald A. Karwoski, Ronald Walker, Bennett A. Landman, Ryan D. Clay, Brian J. Bartholmai, Srinivasan Rajagopalan, Tobias Peikert, Pierre P. Massion, Fabien Maldonado, Jie Tian.


Lung adenocarcinoma (ADC), the most common lung cancer type, is recognized increasingly as a disease spectrum. To guide individualized patient care, a non-invasive means of distinguishing indolent from aggressive ADC subtypes is needed urgently. Computer-Aided Nodule Assessment and Risk Yield (CANARY) is a novel computed tomography (CT) tool that characterizes early ADCs by detecting nine distinct CT voxel classes, representing a spectrum of lepidic to invasive growth, within an ADC. CANARY characterization has been shown to correlate with ADC histology and patient outcomes. This study evaluated the inter-observer variability of CANARY analysis. Three novice observers segmented and analyzed independently 95 biopsy-confirmed lung ADCs from Vanderbilt University Medical Center/Nashville Veterans Administration Tennessee Valley Healthcare system (VUMC/TVHS) and the Mayo Clinic (Mayo). Inter-observer variability was measured using intra-class correlation coefficient (ICC). The average ICC for all CANARY classes was 0.828 (95% CI 0.76, 0.895) for the VUMC/TVHS cohort, and 0.852 (95% CI 0.804, 0.901) for the Mayo cohort. The most invasive voxel classes had the highest ICC values. To determine whether nodule size influenced inter-observer variability, an additional cohort of 49 sub-centimeter nodules from Mayo were also segmented by three observers, with similar ICC results. Our study demonstrates that CANARY ADC classification between novice CANARY users has an acceptably low degree of variability, and supports the further development of CANARY for clinical application.

Partial Text

Now that screening for lung cancer is nationally recommended in most guidelines, the incidence of early lung cancer detection is likely to rise [1]. While this offers a remarkable opportunity to intervene early in the disease course, individualized management of lung cancer therapy will require appropriate risk stratification. Given our evolving knowledge of lung cancer, and the increasingly frequent radiologic detection of tumors that are more indolent than their clinically detected counterparts, over-diagnosis and over-treatment of clinically inconsequential disease are considerable problems. An estimated 20% of cancers diagnosed during the National Lung Screening Trial (NLST) were felt to be slow growing and clinically insignificant, and nearly all of those cancers belonged to the adenocarcinoma (ADC) classification [2–4]. Lung ADC is increasingly recognized as a disease spectrum with varying degrees of aggressiveness, ranging from minimally invasive adenocarcinoma (MIA) and adenocarcinoma in situ (AIS) with nearly 100% post-resection survival to invasive adenocarcinoma (IA) that behaves similarly to other non-small cell lung cancers [5]. Comprehensive semi-quantitative histologic assessment of resected ADCs correlates well with patient outcomes, but cannot by definition be used to guide non-invasive management. Non-invasive characterization of lung ADCs using CT-based quantitative tools could be useful to individualize treatment of lung ADCs.

By characterizing the distributions of nine distinct radiologic voxel classes within ADC nodules, CANARY analysis has been shown to correlate well with histology and patient outcomes in previous studies [6,9], and could prove useful for the objective analysis of these radiologic opacities. Our study demonstrated low inter-observer variability with nodule analysis performed by three novice CANARY users, indicating that the CANARY results published previously could be generalized to users with limited software experience, and establish CANARY as a valuable tool to characterize early lung ADCs.




0 0 vote
Article Rating
Notify of
Inline Feedbacks
View all comments