Research Article: Sentiment in nursing notes as an indicator of out-of-hospital mortality in intensive care patients

Date Published: June 7, 2018

Publisher: Public Library of Science

Author(s): Ian E. R. Waudby-Smith, Nam Tran, Joel A. Dubin, Joon Lee, Peter van Bogaert.


Nursing notes have not been widely used in prediction models for clinical outcomes, despite containing rich information. Advances in natural language processing have made it possible to extract information from large scale unstructured data like nursing notes. This study extracted the sentiment—impressions and attitudes—of nurses, and examined how sentiment relates to 30-day mortality and survival.

This study applied a sentiment analysis algorithm to nursing notes extracted from MIMIC-III, a public intensive care unit (ICU) database. A multiple logistic regression model was fitted to the data to correlate measured sentiment with 30-day mortality while controlling for gender, type of ICU, and SAPS-II score. The association between measured sentiment and 30-day mortality was further examined in assessing the predictive performance of sentiment score as a feature in a classifier, and in a survival analysis for different levels of measured sentiment.

Nursing notes from 27,477 ICU patients, with an overall 30-day mortality of 11.02%, were extracted. In the presence of known predictors of 30-day mortality, mean sentiment polarity was a highly significant predictor in a multiple logistic regression model (Adjusted OR = 0.4626, p < 0.001, 95% CI: [0.4244, 0.5041]) and led to improved predictive accuracy (AUROC = 0.8189 versus 0.8092, 95% BCI of difference: [0.0070, 0.0126]). The Kaplan Meier survival curves showed that mean sentiment polarity quartiles are positively correlated with patient survival (log-rank test: p < 0.001). This study showed that quantitative measures of unstructured clinical notes, such as sentiment of clinicians, correlate with 30-day mortality and survival, thus can also serve as a predictor of patient outcomes in the ICU. Therefore, further research is warranted to study and make use of the wealth of data that clinical notes have to offer.

Partial Text

Common methods of predicting mortality in the intensive care unit (ICU) often use severity of illness scores (SOI), such as APACHE [1], SAPS [2] or SOFA [3]. These SOI systems are based on coded data of the patient’s demographics, lab test results and vital signs that are now commonly available from a patient’s electronic health record (EHR). However, common EHRs also contain unstructured data (not in any organized forms such as a table) like clinical notes written by clinicians that are often not used in mortality prediction. Previous studies suggest that clinicians are reasonably capable of predicting mortality in the ICU [4, 5], therefore their notes should also offer valuable information about the patient’s health status.

This study included 27,477 patients from the selected ICUs that satisfied all the inclusion/exclusion criteria. The overall 30-day mortality of this cohort is 11.02%. The Spearman rho rank correlation between mean sentiment polarity and SAPS-II was found to be -0.2251 (p < 0.001), while that between mean sentiment subjectivity and SAPS-II was found to be 0.0138 (p = 0.022). The results show that even in the presence of known predictors of 30-day mortality, sentiment scores measured in nursing notes are statistically significant predictors, supported by the multiple logistic regression model (Table 3), and lead to improved mortality predictions as measured by AUROC and AUPRC (Table 4). Additionally, the results show that survival is positively correlated to mean sentiment polarity quartiles (Fig 3) but this relationship is much more limited in the case of mean sentiment subjectivity quartiles. These associations are consistent with a previous finding by McCoy et al. that found sentiment measured in discharge notes is associated with readmissions and mortality risk [8]. Taken together, the results suggest that clinical notes authored by hospital staff are informative and can serve as indicators of clinical outcomes, in addition to the information contained in structured clinical data such as vital signs, lab test results, diagnosis, etc.   Source:


0 0 vote
Article Rating
Notify of
Inline Feedbacks
View all comments