Transformer-based Models for Cardiovascular Disease Predictions from Electronic Health Records: A Systematic Review
DOI:
https://doi.org/10.30871/jaic.v10i1.11899Keywords:
Cardiovascular diseases, Electronic Health Records, Explainable AI, Transformer model, Multimodal dataAbstract
This systematic literature review (SLR) analyses 16 studies published between 2020 and 2025 that applied transformer-based or other machine learning models to predict cardiovascular disease (CVD) using electronic health records (EHRs). Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, the review ensures transparency in the identification, screening, and quality appraisal of eligible studies. The key findings reveal a rapid shift from traditional machine learning models, such as Random Forest, toward transformer architectures like the Bidirectional Encoder Representation from Transformers for Electronic Health Record (BEHRT) and its variants. These models demonstrate a superior discrimination (Area Under Curve:0.84 to 0.93) due to their capacity to model long-term temporal dependencies. Explainable AI (XAI) tools, such as attention visualisation, were frequently employed, yet clinical interpretability and integration into decision support remain underexplored. The review also highlights opportunities in federated and privacy-preserving learning, multimodal data fusion, and hybrid architectures that integrate transformers with traditional machine learning methods. This review addresses a gap in the past literature by being the first SLR to compare transformer variants for the prediction of CVDs. Other SLRs examined general CVD risk models, but the present SLR analyses interpretability, external validation and methodological limitations to transformer models. The findings of the recent SLR reported challenges that include data-shift limitations, model-poor population generalisation and their limitations to clinical adoption, which highlights the need for more evaluation protocols and clinicians’ interpretability frameworks.
Downloads
References
[1] B. Chong et al., “Global burden of cardiovascular diseases: projections from 2025 to 2050,” Eur. J. Prev. Cardiol., vol. 00, no. 0, pp. 1–15, 2024, doi: 10.1093/eurjpc/zwae281.
[2] World Health Organization, World health sWORLD HEALTH ORGANIZATION - World health statistics 2024. ISBN 9789240094703. tatistics 2024. 2024.
[3] E. Antikainen et al., “Transformers for cardiac patient mortality risk prediction from heterogeneous electronic health records,” Sci. Rep., vol. 13, no. 1, pp. 1–10, 2023, doi: 10.1038/s41598-023-30657-1.
[4] G. Pallavi, “QFM-BioPred : Quantum Fusion Model for Bioactivity Prediction in Cardiovascular Disease Drug Discovery,” no. September, pp. 1–12, 2025, doi: 10.47852/bonviewJCCE52025138.
[5] J. J. Rose et al., “Cardiopulmonary Impact of Electronic Cigarettes and Vaping Products: A Scientific Statement from the American Heart Association,” Circulation, vol. 148, no. 8, pp. 703–728, 2023, doi: 10.1161/CIR.0000000000001160.
[6] M. Di Cesare et al., “The Heart of the World,” Glob. Heart, vol. 19, no. 1, 2024, doi: 10.5334/gh.1288.
[7] M. Sibindi, S. Sibanda, J. Luke, and C. Mugwanda, “A Predictive Model for Personalized Healthcare Management for Patients with Chronic Diseases,” no. July, 2024, doi: 10.46254/EU07.20240164.
[8] T. Liu, A. J. Krentz, Z. Huo, and V. Ćurčin, “Opportunities and Challenges of Cardiovascular Disease Risk Prediction for Primary Prevention Using Machine Learning and Electronic Health Records: A Systematic Review,” Rev. Cardiovasc. Med., vol. 26, no. 4, 2025, doi: 10.31083/RCM37443.
[9] T. Liu, A. Krentz, L. Lu, and V. Curcin, “Machine learning based prediction models for cardiovascular disease risk using electronic health records data: systematic review and meta-analysis,” Eur. Hear. J. - Digit. Heal., vol. 6, no. 1, pp. 7–22, 2025, doi: 10.1093/ehjdh/ztae080.
[10] N. W.C Mukura and B. Ndlovu, “Performance Evaluation of Artificial Intelligence in Decision Support System for Heart Disease Risk Prediction,” no. Who 2018, pp. 83–93, 2023, doi: 10.46254/ap04.20230043.
[11] Y. Cai et al., “Artificial intelligence in the risk prediction models of cardiovascular disease and development of an independent validation screening tool: a systematic review,” BMC Med., vol. 22, no. 1, pp. 1–18, 2024, doi: 10.1186/s12916-024-03273-7.
[12] R. Nadarajah et al., “Prediction models for heart failure in the community: A systematic review and meta-analysis,” Eur. J. Heart Fail., vol. 25, no. 10, pp. 1724–1738, 2023, doi: 10.1002/ejhf.2970.
[13] M. Odgaard, K. Klein, S. M. Thysen, E. Jimenez-Solem, M. Sillesen, and M. Nielsen, “CORE-BEHRT: A Carefully Optimized and Rigorously Evaluated BEHRT,” Proc. Mach. Learn. Res., vol. 252, pp. 1–33, 2024.
[14] Y. Li et al., “Hi-BEHRT: Hierarchical Transformer-Based Model for Accurate Prediction of Clinical Events Using Multimodal Longitudinal Electronic Health Records,” IEEE J. Biomed. Heal. Informatics, vol. 27, no. 2, pp. 1106–1117, 2023, doi: 10.1109/JBHI.2022.3224727.
[15] S. Rao et al., “Refined selection of individuals for preventive cardiovascular disease treatment with a transformer-based risk model,” Lancet Digit. Heal., vol. 7, no. 6, p. 100873, 2025, doi: 10.1016/j.landig.2025.03.005.
[16] R. Chen and A. Perotte, “Pang21a,” pp. 239–251, 2021.
[17] L. Rasmy, Y. Xiang, Z. Xie, C. Tao, and D. Zhi, “Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction,” npj Digit. Med., vol. 4, no. 1, 2021, doi: 10.1038/s41746-021-00455-y.
[18] Y. Li et al., “BEHRT: Transformer for Electronic Health Records,” Sci. Rep., vol. 10, no. 1, pp. 1–12, 2020, doi: 10.1038/s41598-020-62922-y.
[19] M. Mbaye and M. Danziger, “Multimodal BEHRT : Transformers for Multimodal Electronic Health Records to predict breast cancer prognosis,” 2024.
[20] S. Rao et al., “Targeted-BEHRT: Deep Learning for Observational Causal Inference on Longitudinal Electronic Health Records,” IEEE Trans. Neural Networks Learn. Syst., vol. 35, no. 4, pp. 5027–5038, 2024, doi: 10.1109/TNNLS.2022.3183864.
[21] M. J. Page et al., “The PRISMA 2020 statement: an updated guideline for reporting systematic reviews,” 2021, doi: 10.1136/bmj.n71.
[22] M. Milne-Ives, E. Selby, B. Inkster, C. Lam, and E. Meinert, “Artificial intelligence and machine learning in mobile apps for mental health: A scoping review,” PLOS Digit. Heal., vol. 1, no. 8, pp. 1–13, 2022, doi: 10.1371/journal.pdig.0000079.
[23] F. Andreotti et al., “Prediction of the onset of cardiovascular diseases from electronic health records using multi-task gated recurrent units,” 2020.
[24] C. Li et al., “Improving cardiovascular risk prediction through machine learning modelling of irregularly repeated electronic health records,” Eur. Hear. J. - Digit. Heal., vol. 5, no. 1, pp. 30–40, 2024, doi: 10.1093/ehjdh/ztad058.
[25] O. Ben Shoham and N. Rappoport, “Federated learning of medical concepts embedding using BEHRT,” JAMIA Open, vol. 7, no. 4, pp. 1–10, 2024, doi: 10.1093/jamiaopen/ooae110.
[26] R. Tang et al., “Embedding Electronic Health Records to Learn BERT-based Models for Diagnostic Decision Support,” Proc. - 2021 IEEE 9th Int. Conf. Healthc. Informatics, ISCHI 2021, pp. 311–319, 2021, doi: 10.1109/ICHI52183.2021.00055.
[27] K. Lee, S. W. Oh, S. H. Kim, T. Ko, and I. Y. Choi, “Machine Learning-Based Predictive Models for Early Detection of Cardiovascular Diseases: A Study Utilizing Patient Samples from a Tertiary Health Promotion Center in Korea,” Stud. Health Technol. Inform., vol. 316, no. Ml, pp. 710–711, 2024, doi: 10.3233/SHTI240512.
[28] K. K. Kırboğa and E. U. Küçüksille, “Identifying Cardiovascular Disease Risk Factors in Adults with Explainable Artificial Intelligence,” Anatol. J. Cardiol., vol. 27, no. 11, pp. 657–663, 2023, doi: 10.14744/AnatolJCardiol.2023.3214.
[29] Sadia Latif, Sami Ullah, Aafia Latif, Ghazanfar Ali, Muhammad Hassnain Azhar, and Salman Ali, “Predictive Modeling of Cardiovascular Disease Using Machine Learning Approach,” Kashf J. Multidiscip. Res., vol. 2, no. 02, pp. 207–232, 2025, doi: 10.71146/kjmr288.
[30] A. Alqahtani, S. Alsubai, M. Sha, L. Vilcekova, and T. Javed, “Cardiovascular Disease Detection using Ensemble Learning,” Comput. Intell. Neurosci., vol. 2022, 2022, doi: 10.1155/2022/5267498.
[31] A. H. Elmi, A. Abdullahi, and M. A. Barre, “A machine learning approach to cardiovascular disease prediction with advanced feature selection,” Indones. J. Electr. Eng. Comput. Sci., vol. 33, no. 2, pp. 1030–1041, 2024, doi: 10.11591/ijeecs.v33.i2.pp1030-1041.
[32] Q. Huang et al., “Characterisation of cardiovascular disease (CVD) incidence and machine learning risk prediction in middle-aged and elderly populations: data from the China health and retirement longitudinal study (CHARLS),” BMC Public Health, vol. 25, no. 1, 2025, doi: 10.1186/s12889-025-21609-7.
[33] R. Chen and A. Perotte, “CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks,” Mach. Learn. Heal., vol. 158, pp. 239–251, 2021.
[34] H. A. Salman, A. Kalakech, and A. Steiti, “Random Forest Algorithm Overview,” vol. 2024, pp. 69–79, 2024.
[35] K. Budholiya, S. K. Shrivastava, and V. Sharma, “An optimized XGBoost based diagnostic system for effective prediction of heart disease,” J. King Saud Univ. - Comput. Inf. Sci., vol. 34, no. 7, pp. 4514–4523, 2022, doi: 10.1016/j.jksuci.2020.10.013.
[36] S. Hadebe, B. Ndlovu, and K. Maguraushe, “Managing Diabetes Using Machine Learning and Digital Twins,” Indones. J. Innov. Appl. Sci., vol. 5, no. 2, pp. 145–162, 2025, doi: 10.47540/ijias.v5i2.1981.
[37] C. A. Siebra, M. Kurpicz-Briki, and K. Wac, Transformers in health: a systematic review on architectures for longitudinal data analysis, vol. 57, no. 2. Springer Netherlands, 2024.
[38] T. Lin, Y. Wang, X. Liu, and X. Qiu, “A survey of transformers,” AI Open, vol. 3, pp. 111–132, Jan. 2022, doi: 10.1016/J.AIOPEN.2022.10.001.
[39] A. C. J. W. Janssens and F. K. Martens, “Reflection on modern methods: Revisiting the area under the ROC Curve,” Int. J. Epidemiol., vol. 49, no. 4, pp. 1397–1403, 2020, doi: 10.1093/ije/dyz274.
[40] P. Dubey and P. Dubey, “Advancing CVD Risk Prediction with Transformer Architectures and Statistical Risk Factor Filtering,” 2025.
[41] A. Subbaswamy and S. Saria, “From development to deployment: dataset shift, causality, and shift-stable models in health AI,” Biostatistics, vol. 21, no. 2, pp. 345–352, 2020, doi: 10.1093/biostatistics/kxz041.
[42] Y. Li et al., “Validation of risk prediction models applied to longitudinal electronic health record data for the prediction of major cardiovascular events in the presence of data shifts,” Eur. Hear. J. - Digit. Heal., vol. 3, no. 4, pp. 535–547, 2022, doi: 10.1093/ehjdh/ztac061.
[43] M. Frasca, D. La Torre, G. Pravettoni, and I. Cutica, “Explainable and interpretable artificial intelligence in medicine: a systematic bibliometric review,” Discov. Artif. Intell., vol. 4, no. 1, 2024, doi: 10.1007/s44163-024-00114-7.
[44] H. N. Cho et al., “Explainable predictions of a machine learning model to forecast the postoperative length of stay for severe patients: machine learning model development and evaluation,” BMC Med. Inform. Decis. Mak., vol. 24, no. 1, pp. 1–16, 2024, doi: 10.1186/s12911-024-02755-1.
[45] Z. Revesai, K. Maguraushe, B. Ndlovu, and S. Dube, Predictive Modeling and Risk Assessment of Monkeypox Transmission Using Bayesian Networks : An Interpretable Machine Learning Approach. Springer Nature Singapore, 2026.
[46] O. Mabikwa, B. Ndlovu, and K. Maguraushe, “A Comparative Analysis of Machine Learning Techniques and Explainable AI on Voice Biomarkers for Effective Parkinson ’ s Disease Prediction,” vol. 7, no. 3, pp. 2196–2228, 2025, doi: 10.51519/journalisi.v7i3.1172.
[47] B. Ndlovu, K. Maguraushe, and O. Mabikwa, “Machine Learning and Explainable AI for Parkinson’s Disease Prediction: A Systematic Review,” Indones. J. Comput. Sci., vol. 14, no. 2, 2025, doi: https://doi.org/10.33022/ijcs.v14i2.4837.
[48] T. Ngwazi and B. Ndlovu, “Early Detection of Diabetic Retinopathy Through Explainable AI Models : A Systematic Review,” Int. J. Informatics Dev., vol. 14, no. 2, pp. 616–628, 2025, doi: 10.14421/ijid.2025.5200.
[49] E. H. Houssein, R. E. Mohamed, G. Hu, and A. A. Ali, “Adapting transformer ‑ based language models for heart disease detection and risk factors extraction,” J. Big Data, 2024, doi: 10.1186/s40537-024-00903-y.
[50] M. Farhadizadeh et al., “A systematic review of challenges and proposed solutions in modeling multimodal data,” pp. 1–46, 2025.
[51] K. I. E. Snell et al., “External validation of prognostic models predicting pre-eclampsia : individual participant data meta-analysis,” pp. 1–18, 2020.
[52] K. I. E. Snell et al., “External validation of clinical prediction models: simulation-based sample size calculations were more reliable than rules-of-thumb,” J. Clin. Epidemiol., vol. 135, pp. 79–89, 2021, doi: 10.1016/j.jclinepi.2021.02.011.
[53] Ghassemi et al., “A Review of Challenges and Opportunities in Machine Learning for Health.,” AMIA Jt. Summits Transl. Sci. proceedings. AMIA Jt. Summits Transl. Sci., vol. 2020, pp. 191–200, 2020.
[54] A. Foryciarz, S. R. Pfohl, B. Patel, and N. Shah, “Evaluating algorithmic fairness in the presence of clinical guidelines: The case of atherosclerotic cardiovascular disease risk estimation,” BMJ Heal. Care Informatics, vol. 29, no. 1, 2022, doi: 10.1136/bmjhci-2021-100460.
[55] W. Lyu et al., “A Multimodal Transformer: Fusing Clinical Notes with Structured EHR Data for Interpretable In-Hospital Mortality Prediction.,” AMIA ... Annu. Symp. proceedings. AMIA Symp., vol. 2022, no. Mlm, pp. 719–728, 2022.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Onayi Theresa Chikumo, Belinda Ndlovu
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) ) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
