A survey of automatic rapid meta-analysis: a rapid systematic review
Keywords:
Survey, automatic rapid meta-analysis, AI-powered meta-analysis, AI-powered rapid meta-analysis, Large language Models, rapid systematic reviewAbstract
Systematic reviews and meta-analyses are highly accepted approaches in health science research. The research methodology has standardized thus makes them reliable to analyze the overall picture of various research findings. However, these types of research generally take a long time to produce results and often have high costs. Even with the development of rapid literature review guidelines (rapid review), there is still the problem of not being able to collect knowledge and make it ready for use in a timely manner. Therefore, many automatic tools for systematic literature reviews, meta-analyses have been developed. However, these automations have also been challenging from the past. These tools still not possible to integrate every step of these type of research although automated rapid meta-analysis (RMA) had showed its feasibility. There are also restrictions on access for general researchers due to commercial development of applications/platforms. This rapid systematic review aimed to explore the available platforms that fully integrate the steps of a systematic
literature review to automated meta-analysis. Recently, increasing interest in the use of large language models (LLM) had been widely mentioned, baseline information of all the platforms had been collected to facilitate researchers try using them, to find the right platform for their domain and to serve as a guideline for developing research in this area.
The survey found that the development and the application of automation in medical research still faced many challenges in achieving widespread and reliable use and had not yet reached for fully integration or automating the process of meta-analysis in a seamless way. Therefore, additional studies are needed in the future to develop, integrate and validate the effectiveness of these tools.
References
Lau J. Editorial: Systematic review automation thematic series. Syst Rev. 2019;8:70.
Mutinda FM, Yada S, Wakamiya S, Aramaki E. AUTOMETA: Automatic meta-analysis system employing Natural Language Processing. MEDINFO 2021 Open access by IMIA and IOS Press.612-616: doi:10.3233/SHTI220150N12
Queiros L, Mearns E.S, McCarvil M.S, Alarcão J, Garcia M.J, Abògúnr̀in S. Is artificial intelligence replacing humans in systematic literature reviews? a systematic literature review. Research Square 2023 Oct:1-21 Available from: https://doi.org/10.21203/rs.3.rs-3119345/v1
Tricco AC, Langlois EV, Straus SE, editors. Rapid reviews to strengthen health policy and systems: a practical guide. Geneva: World Health Organization; 2017.
Garritty C, Gartlehner G, Kamel C, King VJ, Nussbaumer-Streit B, Stevens A, Hamel C, Affengruber L. Cochrane Rapid Reviews. Interim Guidance from the Cochrane Rapid Reviews Methods Group. March 2020.
Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, Perrier L, Hutton B, Moher D, Straus SE. A scoping review of rapid review methods. BMC Medicine. 2015;13:224: DOI 10.1186/s12916-015-0465-6
Robinson E, Jones A, Lesser I, Daly M. International estimates of intended uptake and refusal of COVID-19 vaccines: a rapid systematic review and meta-analysis of large nationally representative samples. medRxiv December 2020. Available from: https://doi.org/10.1101/2020.12.01.20241729
Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E. Systematic review automation technologies. Systematic Reviews. 2014;3:74.
Marshall JI, Wallace BC. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Systematic Reviews. 2019;8:163.
O’Connor AM, Tsafnat G, Gilbert SB , Thayer KA , Shemilt I, Thomas J, Glasziou P, Wolfe MS. Still moving toward automation of the systematic review process: a summary of discussions at the third meeting of the International Collaboration for Automation of Systematic Reviews (ICASR). Systematic Reviews 2019;8:57. Available from: https://doi.org/10.1186/s13643-019-0975-y
van Dinter R, Tekinerdogan B, Catal C. Automation of systematic literature reviews: a systematic literature review. Information and Software Technology. 2021;1:136.
Christopoulou SC. Machine learning tools and platforms in clinical trial outputs to support evidence-based health informatics: a rapid review of the literature. BioMedInformatics. 2022;3:511–27.
Beller E, Clark J, Tsafnat G, Adams C, Diehl H, Lund H, Ouzzani M, Thayer K, Thomas J, Turner T, Xia J, Robinson K, Glasziou P. On behalf of the founding members of the ICASR group. Making progress with the automation of systematic reviews: principles of the International Collaboration for the Automation of Systematic Reviews (ICASR). Systematic Reviews. 2018;7:77
Michelson M. Automating meta-analyses of randomized clinical trials: A first look. expanding the boundaries of health informatics using AI. Papers from the 2014 AAAI Fall Symposium p.22-26
Michelson M, Chow T, Martin NA, Ross M, Ying ATQ, Minton S. Artificial intelligence for rapid meta-analysis: case study on ocular toxicity of hydroxychloroquine. J Med Internet Res. 2020;17:e20007.
Zhang X, Wang C, Yao Y, Sun W, Guo Y, Ma L, Lu X, Deng X. Construction of a meta-evidence prototype database of traditional Chinese medicine splenogastric diseases and its application in an automatic meta-analysis system. Hindawi Evidence-Based Complementary and Alternative Medicine Volume 2022:1-8. Available from: https://doi.org/10.1155/2022/693352
Meskó B, Topol E.J. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. Digital Med. 2023;6:120.
Kisley M. ChatGPT for Research: Do’s and Don’ts in 2024. Available from: https://litmaps.substack.com/p/chatgpt-for-research-in-2024-dos
Orel E, Ciglenecki I, Thiabaud A, Temerev A, Calmy A, Keiser O, et al. An automated literature review tool (LiteRev) for streamlining and accelerating research using Natural Language Processing and Machine Learning: descriptive performance evaluation study. J Med Internet Res. 2023;25:e39736.
Lam Hoai XL, Simonart T. Comparing meta-analyses with ChatGPT in the evaluation of the effectiveness and tolerance of systemic therapies in moderate-to-severe plaque psoriasis. J Clin Med. 2023;12:5410.
Miller LE, Bhattacharyya D, Miller VM, Bhattacharyya M. Recent trend in artificial intelligence-assisted biomedical publishing: a quantitative bibliometric analysis. Cureus. 2023;15:e39224.
McGenity C, Clarke EL, Jennings C, Matthews G, Cartlidge C, Freduah-Agyemang H, et al. Artificial intelligence in digital pathology: a diagnostic test accuracy systematic review and metaanalysis [Internet]. arXiv; 2023 [cited 2024 Feb 8]. Available from: http://arxiv.org/abs/2306.07999
van Dijk SHB, Brusse-Keizer MGJ, Bucsán CC, van der Palen J, Doggen CJM, Lenferink A. Artificial intelligence in systematic reviews: promising when appropriately used. BMJ Open. 2023;13:e072254.
de la Torre-López J, Ramírez A, Romero JR. Artificial intelligence to automate the systematic review of scientific literature. Computing. 2023;105:2171–94.
Šuster S, Baldwin T, Lau JH, Yepes AJ, Iraola DM, Otmakhova Y, et al. Automating quality assessment of medical evidence in systematic reviews: model development and validation study. J Med Internet Res. 2023;25:e35568.
Schmidt L, Finnerty Mutlu AN, Elmore R, Olorisade BK, Thomas J, Higgins JPT. Data extraction methods for systematic review (semi) automation: Update of a living systematic review. F1000Res. 2021;10:401.
Panayi A, Ward K, Benhadji-Schaff A, Ibanez-Lopez AS, Xia A, Barzilay R. Evaluation of a prototype machine learning tool to semi-automate data extraction for systematic literature reviews. Systematic Reviews. 2023;12:187.
Hamel C, Hersi M, Kelly SE, Tricco AC, Straus S, Wells G, et al. Guidance for using artificial intelligence for title and abstract screening while conducting knowledge syntheses. BMC Medical Res Methodol. 2021;21:285.
Cowie K, Rahmatullah A, Hardy N, Holub K, Kallmes K. Web-Based Software Tools for Systematic Literature Review in Medicine: Systematic Search and Feature Analysis. JMIR Med Inform. 2022;10:e33219.
Hulsen T. Literature analysis of artificial intelligence in biomedicine. Ann Transl Med. 2022;10:1284.
Lange T, Schwarzer G, Datzmann T, Binder H. Machine learning for identifying relevant publications in updates of systematic reviews of diagnostic test studies. Res Synthesis Methods. 2021;12:506–15.
Jaspers S, De Troyer E, Aerts M. Machine learning techniques for the automation of literature reviews and systematic reviews in EFSA. EFSA Supporting Publications. 2018;15:1427E.
Abba K, Deeks JJ, Olliaro PL, Naing CM, Jackson SM, Takwoingi Y, et al. Rapid diagnostic tests for diagnosing uncomplicated P. falciparum malaria in endemic countries. Cochrane Infectious Diseases Group, editor. Cochrane Database of Systematic Reviews [Internet]. 2011 Jul 6 [cited 2024 Feb 8]; Available from: https://doi.wiley.com/10.1002/14651858.CD008122.pub2
Shelmerdine SC, Arthurs OJ, Denniston A, Sebire NJ. Review of study reporting guidelines for clinical studies using artificial intelligence in healthcare. BMJ Health Care Inform. 2021;28:e100385.
Scott AM, Forbes C, Clark J, Carter M, Glasziou P, Munn Z. Systematic review automation tools improve efficiency but lack of knowledge impedes their adoption: a survey. J Clin Epidemiol. 2021;138:80–94.
Santos ÁO dos, da Silva ES, Couto LM, Reis GVL, Belo VS. The use of artificial intelligence for automating or semi-automating biomedical literature analyses: A scoping review. J Biomed. Informatics. 2023;142:104389.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Journal of Medical Globalization
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
