Date Published: November 11, 2016
Publisher: Public Library of Science
Author(s): Brandon K. Peoples, Stephen R. Midway, Dana Sackett, Abigail Lynch, Patrick B. Cooney, Lutz Bornmann.
The relationship between traditional metrics of research impact (e.g., number of citations) and alternative metrics (altmetrics) such as Twitter activity are of great interest, but remain imprecisely quantified. We used generalized linear mixed modeling to estimate the relative effects of Twitter activity, journal impact factor, and time since publication on Web of Science citation rates of 1,599 primary research articles from 20 ecology journals published from 2012–2014. We found a strong positive relationship between Twitter activity (i.e., the number of unique tweets about an article) and number of citations. Twitter activity was a more important predictor of citation rates than 5-year journal impact factor. Moreover, Twitter activity was not driven by journal impact factor; the ‘highest-impact’ journals were not necessarily the most discussed online. The effect of Twitter activity was only about a fifth as strong as time since publication; accounting for this confounding factor was critical for estimating the true effects of Twitter use. Articles in impactful journals can become heavily cited, but articles in journals with lower impact factors can generate considerable Twitter activity and also become heavily cited. Authors may benefit from establishing a strong social media presence, but should not expect research to become highly cited solely through social media promotion. Our research demonstrates that altmetrics and traditional metrics can be closely related, but not identical. We suggest that both altmetrics and traditional citation rates can be useful metrics of research impact.
Scientific writing is at the core of numerous professions, including academics, industry, government and agency work, and others. The success with which we often measure the breadth and impact of an individual’s written output forms the basis for job promotion, future research, products, and other important outputs. Accordingly, researchers are under constant pressure to boost traditional metrics of research output—namely the h-index, which accounts for an author’s number of publications (research output) and citation rates (research impact/quality). Therefore, an accurate understanding of the dynamics that make some scientific articles successful (where success is defined by future citations) is insightful for both individual scientists and research organizations.
We collected Twitter activity (defined here to include three metrics: number of tweets, number of users, twitter reach; see Table 1) and citation data on articles from twenty journals that publish only ecological research (Table 2). We excluded general scientific journal articles that include ecology as a disciplinary subset (e.g. Science or Nature), and ecology journals that publish only reviews and/or nontechnical pieces (e.g. Frontiers in Ecology and the Environment and Trends in Ecology and Evolution). We selected journals to represent a range of impact factors (IF, identified from Thompson-Reuters 2014 Journal Citation Reports® database). We used 5-year IF as a metric of journal impact because it is more stable than yearly IFs, and is representative of most traditional measures of journal impact. We excluded journals with IF < 3.0 to minimize zero-inflation in the distribution of citation rates (i.e. many journal articles with zero citations). This cutoff value also serves to reduce potential unwanted variation caused by discipline-specific differences in Twitter activity: higher-impact ecology journals feature only general ecology, while discipline-specificity increases as impact factor decreases. Lower-impact journals typically focus more on specific processes, taxa, or systems, and are much more heterogeneous in many aspects of publication (e.g. article promotion, publication time, research timeliness, etc.). We acknowledge that this approach may bias some of our results by inflating the number of papers that have been mentioned on Twitter , or by missing a few key patterns within outlying journals. However, we are confident that our broad coverage of twenty journals and nearly 1,600 articles helps to ensure that we observed the true underlying patterns in the ecological research. We collected data on articles published from 2012 to 2014 to further minimize zero-inflation in citation distributions, as this allowed time for an article to be cited  and ensured articles published before the common use of Twitter were excluded. Because most social activity surrounding an articles occurs within a week of its initial publication, our time window also helped to reduce the possibility of Twitter use data changing dramatically throughout the data collection period. We collected a total of 1,599 primary research articles among the twenty ecology journals over three years. Twitter activity and citation rates were variable among articles (Table 2). Over a fourth (28%, n = 442) of articles had no Twitter activity, and another 17% (266) had only one tweet. Number of tweets and Twitter reach were moderately correlated (r = 0.65); Twitter reach of single-tweet articles ranged from 0 (one tweet from an account with zero followers) to 10,939 users. However, number of users was strongly correlated with number of tweets (r = 0.97); we thus excluded number of users from GLMM analysis. This study provides evidence that Twitter activity associated with primary ecological research articles is significantly and positively associated with the number of future citations. Although focusing solely on higher-impact ecology journals, this represents the first study to compare the relative effects of social media activity, journal impact factor, and time since publication on citation rates of research from any discipline. In doing so, we found that the role of journal IF can be strong but variable, and that the effect of time since publication can outweigh both Twitter activity and IF. Our inclusion of multiple journals demonstrates that these patterns are not specific to any one particular journal, but instead are generalizable across journals within the discipline of ecology. Although inference may be constrained within the discipline of ecology, we expect the patterns to hold across other disciplines, as well [14, 29]. Source: http://doi.org/10.1371/journal.pone.0166570