TY - JOUR
T1 - Reporting guidelines for clinical trial reports for interventions involving artificial intelligence
T2 - the CONSORT-AI extension
AU - Liu, Xiaoxuan
AU - Cruz Rivera, Samantha
AU - Moher, David
AU - Calvert, Melanie J.
AU - Denniston, Alastair K.
AU - Chan, An Wen
AU - Darzi, Ara
AU - Holmes, Christopher
AU - Yau, Christopher
AU - Ashrafian, Hutan
AU - Deeks, Jonathan J.
AU - Ferrante di Ruffano, Lavinia
AU - Faes, Livia
AU - Keane, Pearse A.
AU - Vollmer, Sebastian J.
AU - Lee, Aaron Y.
AU - Jonas, Adrian
AU - Esteva, Andre
AU - Beam, Andrew L.
AU - Chan, An Wen
AU - Panico, Maria Beatrice
AU - Lee, Cecilia S.
AU - Haug, Charlotte
AU - Kelly, Christopher J.
AU - Mulrow, Cynthia
AU - Espinoza, Cyrus
AU - Fletcher, John
AU - Paltoo, Dina
AU - Manna, Elaine
AU - Price, Gary
AU - Collins, Gary S.
AU - Harvey, Hugh
AU - Matcham, James
AU - Monteiro, Joao
AU - ElZarrad, M. Khair
AU - Ferrante di Ruffano, Lavinia
AU - Oakden-Rayner, Luke
AU - McCradden, Melissa
AU - Keane, Pearse A.
AU - Savage, Richard
AU - Golub, Robert
AU - Sarkar, Rupa
AU - Rowley, Samuel
AU - The SPIRIT-AI and CONSORT-AI Working Group
AU - SPIRIT-AI and CONSORT-AI Steering Group
AU - Vaillant, Michel
N1 - Funding Information:
We thank the participants who were involved in the Delphi study and Pilot study (Supplementary Note); E. Marston (University of Birmingham, UK) for strategic support; and C. Radovanovic (University Hospitals Birmingham NHS Foundation Trust, UK) and A. Walker (University of Birmingham, UK) for administrative support. The views expressed in this publication are those of the authors, Delphi participants and stakeholder participants and may not represent the views of the broader stakeholder group or host institution. This work was funded by a Wellcome Trust Institutional Strategic Support Fund: Digital Health Pilot Grant, Research England (part of UK Research and Innovation), Health Data Research UK and the Alan Turing Institute. The study was sponsored by the University of Birmingham, UK. The study funders and sponsors had no role in the design and conduct of the study; collection, management, analysis and interpretation of the data; preparation, review or approval of the manuscript; or decision to submit the manuscript for publication. M.J.C. is a National Institute for Health Research (NIHR) Senior Investigator and receives funding from the National Institute for Health Research (NIHR) Birmingham Biomedical Research Centre; the NIHR Surgical Reconstruction and Microbiology Research Centre and NIHR ARC West Midlands at the University of Birmingham and University Hospitals Birmingham NHS Foundation Trust; Health Data Research UK; Innovate UK (part of UK Research and Innovation); the Health Foundation; Macmillan Cancer Support; and UCB Pharma. A.D. and J.D. are also NIHR Senior Investigators. The views expressed in this article are those of the author(s) and not necessarily those of the NIHR, or the Department of Health and Social Care. S.J.V. receives funding from the Engineering and Physical Sciences Research Council, UK Research and Innovation (UKRI), Accenture, Warwick Impact Fund, Health Data Research UK and European Regional Development Fund. S.R. is an employee of the Medical Research Council (UKRI). D.M. is supported by a University of Ottawa Research Chair. M.K.E. is supported by the U.S. Food and Drug Administration (FDA), and D.P. is supported in part by the Office of the Director at the National Library of Medicine (NLM), US National Institutes of Health (NIH). A.B. is supported by a National Institutes of Health (NIH) award 7K01HL141771-02. This article may not be consistent with NIH and/or FDA’s views or policies. It reflects only the views and opinions of the authors.
Publisher Copyright:
© 2020, The Author(s).
PY - 2020/9/1
Y1 - 2020/9/1
N2 - The CONSORT 2010 statement provides minimum guidelines for reporting randomized trials. Its widespread use has been instrumental in ensuring transparency in the evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate impact on health outcomes. The CONSORT-AI (Consolidated Standards of Reporting Trials–Artificial Intelligence) extension is a new reporting guideline for clinical trials evaluating interventions with an AI component. It was developed in parallel with its companion statement for clinical trial protocols: SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 29 candidate items, which were assessed by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a two-day consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The CONSORT-AI extension includes 14 new items that were considered sufficiently important for AI interventions that they should be routinely reported in addition to the core CONSORT 2010 items. CONSORT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human–AI interaction and provision of an analysis of error cases. CONSORT-AI will help promote transparency and completeness in reporting clinical trials for AI interventions. It will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.
AB - The CONSORT 2010 statement provides minimum guidelines for reporting randomized trials. Its widespread use has been instrumental in ensuring transparency in the evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate impact on health outcomes. The CONSORT-AI (Consolidated Standards of Reporting Trials–Artificial Intelligence) extension is a new reporting guideline for clinical trials evaluating interventions with an AI component. It was developed in parallel with its companion statement for clinical trial protocols: SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 29 candidate items, which were assessed by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a two-day consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The CONSORT-AI extension includes 14 new items that were considered sufficiently important for AI interventions that they should be routinely reported in addition to the core CONSORT 2010 items. CONSORT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human–AI interaction and provision of an analysis of error cases. CONSORT-AI will help promote transparency and completeness in reporting clinical trials for AI interventions. It will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.
UR - http://www.scopus.com/inward/record.url?scp=85090819278&partnerID=8YFLogxK
UR - https://pubmed.ncbi.nlm.nih.gov/32908283
U2 - 10.1038/s41591-020-1034-x
DO - 10.1038/s41591-020-1034-x
M3 - Article
C2 - 32908283
AN - SCOPUS:85090819278
SN - 1078-8956
VL - 26
SP - 1364
EP - 1374
JO - Nature Medicine
JF - Nature Medicine
IS - 9
ER -