Transformer-Based Models for Electronic Health Records and Omics in Healthcare: A Systematic Literature Review
DOI:
https://doi.org/10.30871/jaic.v10i1.11893Keywords:
Artificial Intelligence, Machine Learning, Transformer, Electronic Health Records (EHRs), Electronic Medical Records (EMRs), OmicsAbstract
Electronic Health Records (EHRs) have become central to modern healthcare. The emergence of transformer-based models has profoundly influenced how EHRs are used for modelling complex, longitudinal data. Integration with omics technologies improves the precision of disease identification and risk assessment during modelling. While several reviews have examined transformers in healthcare broadly, a systematic synthesis focused on their architectural design, empirical performance and integration of EHRs with omics data remains limited. This study presents a systematic literature review of transformer-based models applied to electronic health records (EHRs) and omics data, and of their integration into healthcare. Following PRISMA guidelines, peer-reviewed studies were retrieved from IEEE Xplore, ACM Digital Library, PubMed, and ScienceDirect, resulting in 14 eligible empirical studies published between 2020 and 2025. The review analyses transformer architectures, submodules, application domains, comparative performance, interpretability mechanisms, and limitations. Findings indicate that architectural design drives task-specific advantages in disease prediction, phenotyping, medication recommendation, and omics analysis. The integration of self-attention with deep learning, temporal modelling, and a pre-trained biomedical transformer improves performance. However, most studies remain centred on EHR, with limited empirical integration of omics data. Persistent challenges include limited generalisability, high computational cost, data quality issues, and insufficient interpretability for clinical deployment. The primary contribution of this review lies in synthesising architectural trends and methodological gaps. By consolidating current evidence, the study provides clear directions for the development of explainable, generalisable, and multimodal transformer-based systems in precision healthcare.
Downloads
References
[1] F. Ge, X. Yu, X. Li, X. Fan, and Y. Zhao, “Personalized and safe medication recommendation based on convolutional neural network and transformer architecture,” Eng. Appl. Artif. Intell., vol. 161, Dec. 2025, doi: 10.1016/j.engappai.2025.112267.
[2] S. X. E. G. J. K. L. W. S. L. W.-Q. W. P. J. D. M. R. Crosslin, “Transformer patient embedding using electronic health records enables patient stratification and progression analysis,” npj Digit. Med., 2025, doi: 10.1038/s41746-025-01872-z.
[3] G. Chitra and S. M. Basha, “Systematic Literature Review on Deep Learning Techniques in Electronic Health Records,” 2023 4th Int. Conf. Electron. Sustain. Commun. Syst. ICESC 2023 - Proc., no. January, pp. 1365–1371, 2023, doi: 10.1109/ICESC57686.2023.10193025.
[4] R. Sibanda, B. Ndlovu, S. Dube, and K. Maguraushe, “Towards Health 4 . 0 : Blockchain-Based Electronic Health Record for Care Coordination,” pp. 712–720, 2024, doi: 10.34190/ecie.19.1.2606..
[5] M. S. Safarova and I. J. Kullo, “Using the electronic health record for genomics research,” Curr. Opin. Lipidol., vol. 31, no. 2, pp. 85–93, 2020, doi: 10.1097/MOL.0000000000000662.
[6] S. A. Pendergrass and D. C. Crawford, “Using Electronic Health Records To Generate Phenotypes For Research,” Curr. Protoc. Hum. Genet., vol. 100, no. 1, pp. 1–28, 2019, doi: 10.1002/cphg.80.
[7] K. Himavamshi, D. Tejaswini, G. Sethi, V. S. A. Devi, P. Pavani, and S. Hariharan, “Electronic Health Record classification and analysis using NLP Techniques,” E3S Web Conf., vol. 619, 2025, doi: 10.1051/e3sconf/202561903016.
[8] M. R. K. Goyal, “Integrating omics data for personalized medicine in treating psoriasis,” Med. Chem. Res., 2024, doi: 10.1007/s00044-024-03355-4.
[9] Y. Wu, Y. Cheng, X. Wang, J. Fan, and Q. Gao, “Spatial omics: Navigating to the golden era of cancer research,” Clin. Transl. Med., vol. 12, no. 1, 2022, doi: 10.1002/ctm2.696.
[10] Y. Ren et al., “COMET: Benchmark for Comprehensive Biological Multi-omics Evaluation Tasks and Language Models,” pp. 1–29, 2024, [Online]. Available: http://arxiv.org/abs/2412.10347
[11] A. E. Mohr, C. P. Ortega-Santos, C. M. Whisner, J. Klein-Seetharaman, and P. Jasbi, “Navigating Challenges and Opportunities in Multi-Omics Integration for Personalized Healthcare,” Biomedicines, vol. 12, no. 7, 2024, doi: 10.3390/biomedicines12071496.
[12] L. Zhao et al., “DeepOmix: A scalable and interpretable multi-omics deep learning framework and application in cancer survival analysis,” Comput. Struct. Biotechnol. J., vol. 19, pp. 2719–2725, 2021, doi: 10.1016/j.csbj.2021.04.067.
[13] J. Zhao, Q. Feng, and W.-Q. Wei, “Integration of Omics and Phenotypic Data for Precision Medicine.,” Methods Mol. Biol., vol. 2486, pp. 19–35, 2022, doi: 10.1007/978-1-0716-2265-0_2.
[14] M. L. G. G. T. T. Y. J. S. Jia, “Machine learning and multi-omics integration: advancing cardiovascular translational research and clinical practice,” J. Transl. Med., 2025, doi: 10.1186/s12967-025-06425-2.
[15] I. C. Udousoro, “Machine Learning: A Review,” Semicond. Sci. Inf. Devices, vol. 2, no. 2, pp. 5–14, 2020, doi: 10.30564/ssid.v2i2.1931.
[16] S. Hadebe, B. Ndlovu, and K. Maguraushe, “Managing Diabetes Using Machine Learning and Digital Twins,” pp. 145–162, 2025, doi: 10.47540/ijias.v5i2.1981.
[17] I. S. Mangkunegara, Purwono, A. Ma’arif, N. Basil, H. M. Marhoon, and A. N. Sharkawy, “Transformer Models in Deep Learning: Foundations, Advances, Challenges and Future Directions,” Bul. Ilm. Sarj. Tek. Elektro, vol. 7, no. 2, pp. 231–241, 2025, doi: 10.12928/biste.v7i2.13053.
[18] A. Behrouz, P. Zhong, and V. Mirrokni, “Titans: Learning to Memorize at Test Time,” pp. 1–27, 2024, [Online]. Available: http://arxiv.org/abs/2501.00663
[19] A. Mohamed, R. AlAleeli, and K. Shaalan, “Advancing Predictive Healthcare: A Systematic Review of Transformer Models in Electronic Health Records,” Computers, vol. 14, no. 4, pp. 1–28, 2025, doi: 10.3390/computers14040148.
[20] V. A. Batista and A. G. Evsukoff, “Application of Transformers based methods in Electronic Medical Records: A Systematic Literature Review,” pp. 1–28, 2023, [Online]. Available: http://arxiv.org/abs/2304.02768
[21] C. A. Siebra, M. Kurpicz-Briki, and K. Wac, Transformers in health: a systematic review on architectures for longitudinal data analysis, vol. 57, no. 2. Springer Netherlands, 2024. doi: 10.1007/s10462-023-10677-z.
[22] S. Shool, S. Adimi, R. Saboori Amleshi, E. Bitaraf, R. Golpira, and M. Tara, “A systematic review of large language model (LLM) evaluations in clinical medicine,” BMC Med. Inform. Decis. Mak., vol. 25, no. 1, 2025, doi: 10.1186/s12911-025-02954-4.
[23] A. Mohamed, R. AlAleeli, and K. Shaalan, “Advancing Predictive Healthcare: A Systematic Review of Transformer Models in Electronic Health Records,” Computers, vol. 14, no. 4, 2025, doi: 10.3390/computers14040148.
[24] A. Mohamed, R. AlAleeli, and K. Shaalan, “Advancing Predictive Healthcare: A Systematic Review of Transformer Models in Electronic Health Records,” Apr. 01, 2025, Multidisciplinary Digital Publishing Institute (MDPI). doi: 10.3390/computers14040148.
[25] M. J. Page et al., “PRISMA 2020 explanation and elaboration: Updated guidance and exemplars for reporting systematic reviews,” BMJ, vol. 372, 2021, doi: 10.1136/bmj.n160.
[26] T. Dybå and T. Dingsøyr, “Empirical studies of agile software development: A systematic review,” Inf. Softw. Technol., vol. 50, no. 9–10, pp. 833–859, 2008, doi: 10.1016/j.infsof.2008.01.006.
[27] B. Kitchenham, “Kitchenham , B .: Guidelines for performing Systematic Literature Reviews in software engineering . EBSE Technical Report EBSE-2007-01 Guidelines for performing Systematic Literature Reviews in Software Engineering,” Icse, no. January 2007, pp. 1–57, 2007.
[28] S. Keele, “Guidelines for performing systematic literature reviews in software engineering,” Tech. report, Ver. 2.3 EBSE Tech. Report. EBSE, no. October, 2007.
[29] L. ning Zhang, J. wei Liu, Z. yan Song, and X. Zuo, “Universal transformer Hawkes process with adaptive recursive iteration,” Eng. Appl. Artif. Intell., vol. 105, Oct. 2021, doi: 10.1016/j.engappai.2021.104416.
[30] Z. Song, Q. Lu, H. Xu, H. Zhu, D. Buckeridge, and Y. Li, “TimelyGPT: Extrapolatable Transformer Pre-training for Long-term Time-Series Forecasting in Healthcare,” in Proceedings of the 15th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, in BCB ’24. New York, NY, USA: Association for Computing Machinery, 2024. doi: 10.1145/3698587.3701364.
[31] A. D. Diaz Gonzalez, K. S. Hughes, S. Yue, and S. T. Hayes, “Applying BioBERT to Extract Germline Gene-Disease Associations for Building a Knowledge Graph from the Biomedical Literature,” in Proceedings of the 2023 7th International Conference on Information System and Data Mining, in ICISDM ’23. New York, NY, USA: Association for Computing Machinery, 2023, pp. 37–42. doi: 10.1145/3603765.3603771.
[32] R. Miao et al., “An Integrated Multi-omics prediction model for stroke recurrence based on Lnet transformer layer and dynamic weighting mechanism,” Comput. Biol. Med., vol. 179, Sep. 2024, doi: 10.1016/j.compbiomed.2024.108823.
[33] S. Rao et al., “Refined selection of individuals for preventive cardiovascular disease treatment with a transformer-based risk model,” Lancet Digit. Heal., vol. 7, no. 6, Jun. 2025, doi: 10.1016/j.landig.2025.03.005.
[34] Z. Fan, M. Mamouei, Y. Li, S. Rao, and K. Rahimi, “Identification of heart failure subtypes using transformer-based deep learning modelling: a population-based study of 379,108 individuals,” eBioMedicine, vol. 114, Apr. 2025, doi: 10.1016/j.ebiom.2025.105657.
[35] D. M. Jordan, H. M. T. Vy, and R. Do, “A deep learning transformer model predicts high rates of undiagnosed rare disease in large electronic health systems,” medRxiv, p. 2023.12.21.23300393, Dec. 2023, doi: 10.1101/2023.12.21.23300393.
[36] D. Patel et al., “Traditional Machine Learning, Deep Learning, and BERT (Large Language Model) Approaches for Predicting Hospitalizations From Nurse Triage Notes: Comparative Evaluation of Resource Management,” JMIR AI, vol. 3, no. 1, 2024, doi: 10.2196/52190.
[37] M. Tharmakulasingam, W. Wang, M. Kerby, R. La Ragione, and A. Fernando, “TransAMR: An Interpretable Transformer Model for Accurate Prediction of Antimicrobial Resistance Using Antibiotic Administration Data,” IEEE Access, vol. 11, pp. 75337–75350, 2023, doi: 10.1109/ACCESS.2023.3296221.
[38] X. Zhou et al., “CMACF: Transformer-based cross-modal attention cross-fusion model for systemic lupus erythematosus diagnosis combining Raman spectroscopy, FTIR spectroscopy, and metabolomics,” Inf. Process. Manag., vol. 61, no. 6, Nov. 2024, doi: 10.1016/j.ipm.2024.103804.
[39] X. Luo et al., “Applying interpretable deep learning models to identify chronic cough patients using EHR data,” Comput. Methods Programs Biomed., vol. 210, Oct. 2021, doi: 10.1016/j.cmpb.2021.106395.
[40] S. Naveed, M. Husnain, and N. Alsubaie, “HybridDLDR: A hybrid deep learning-based drug resistance prediction system of Glioblastoma (GBM) using molecular descriptors and gene expression data,” Comput. Methods Programs Biomed., vol. 270, Oct. 2025, doi: 10.1016/j.cmpb.2025.108913.
[41] X. Wang et al., “Hierarchical Pretraining on Multimodal Electronic Health Records,” EMNLP 2023 - 2023 Conf. Empir. Methods Nat. Lang. Process. Proc., pp. 2839–2852, 2023, doi: 10.18653/v1/2023.emnlp-main.171.
[42] M. McDermott et al., “A comprehensive EHR timeseries pre-training benchmark,” ACM CHIL 2021 - Proc. 2021 ACM Conf. Heal. Inference, Learn., pp. 257–278, 2021, doi: 10.1145/3450439.3451877.
[43] A. Patharkar, F. Cai, F. Al-Hindawi, and T. Wu, “Predictive modeling of biomedical temporal data in healthcare applications: review and future directions,” Front. Physiol., vol. 15, no. October, pp. 1–24, 2024, doi: 10.3389/fphys.2024.1386760.
[44] J. Zhou et al., “Graph neural networks: A review of methods and applications,” AI Open, vol. 1, no. September 2020, pp. 57–81, 2020, doi: 10.1016/j.aiopen.2021.01.001.
[45] S. R. Choi and M. Lee, “Transformer Architecture and Attention Mechanisms in Genome Data Analysis: A Comprehensive Review,” Biology (Basel)., vol. 12, no. 7, 2023, doi: 10.3390/biology12071033.
[46] L. Tong et al., “Integrating Multi-Omics Data With EHR for Precision Medicine Using Advanced Artificial Intelligence,” IEEE Rev. Biomed. Eng., vol. 17, pp. 80–97, 2024, doi: 10.1109/RBME.2023.3324264.
[47] S. Madan, M. Lentzen, J. Brandt, D. Rueckert, M. Hofmann-Apitius, and H. Fröhlich, “Transformer models in biomedicine,” BMC Med. Informatics Decis. Mak. 2024 241, vol. 24, no. 1, pp. 1–22, Jul. 2024, doi: 10.1186/S12911-024-02600-5.
[48] J. Li et al., “An integrated pipeline for prediction of Clostridioides difficile infection.,” Sci. Rep., vol. 13, no. 1, p. 16532, Oct. 2023, doi: 10.1038/s41598-023-41753-7.
[49] B. Shickel, P. J. Tighe, A. Bihorac, and P. Rashidi, “Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis,” IEEE J. Biomed. Heal. Informatics, vol. 22, no. 5, pp. 1589–1604, 2018, doi: 10.1109/JBHI.2017.2767063.
[50] A. Rajkomar, J. Dean, and I. Kohane, “Machine Learning in Medicine,” N. Engl. J. Med., vol. 380, no. 14, pp. 1347–1358, 2019, doi: 10.1056/nejmra1814259.
[51] Y. Wang et al., “Clinical information extraction applications: A literature review,” J. Biomed. Inform., vol. 77, no. November 2017, pp. 34–49, 2018, doi: 10.1016/j.jbi.2017.11.011.
[52] A. Vaid et al., “Using Deep-Learning Algorithms to Simultaneously Identify Right and Left Ventricular Dysfunction From the Electrocardiogram,” JACC Cardiovasc. Imaging, vol. 15, no. 3, pp. 395–410, Mar. 2022, doi: 10.1016/j.jcmg.2021.08.004.
[53] J. T. VanSchaik, P. Jain, A. Rajapuri, B. Cheriyan, T. P. Thyvalikakath, and S. Chakraborty, “Using transfer learning-based causality extraction to mine latent factors for Sjögren’s syndrome from biomedical literature,” Heliyon, vol. 9, no. 9, Sep. 2023, doi: 10.1016/j.heliyon.2023.e19265.
[54] J. U. Ashish Vaswani, Noam Shazeer, Niki Parmar, “Attention Is All You Need Ashish,” IEEE Ind. Appl. Mag., vol. 8, no. 1, pp. 8–15, 2017, doi: 10.1109/2943.974352.
[55] N. W.C Mukura and B. Ndlovu, “Performance Evaluation of Artificial Intelligence in Decision Support System for Heart Disease Risk Prediction,” no. Who 2018, pp. 83–93, 2023, doi: 10.46254/ap04.20230043.
[56] A. Vaid et al., “Federated learning of electronic health records to improve mortality prediction in hospitalized patients with COVID-19: Machine learning approach,” JMIR Med. Informatics, vol. 9, no. 1, Jan. 2021, doi: 10.2196/24207.
[57] T. Wang et al., “MOGONET integrates multi-omics data using graph convolutional networks allowing patient classification and biomarker identification,” Nat. Commun., vol. 12, no. 1, pp. 1–13, 2021, doi: 10.1038/s41467-021-23774-w.
[58] T. Ching et al., Opportunities and obstacles for deep learning in biology and medicine, vol. 15, no. 141. 2018. doi: 10.1098/rsif.2017.0387.
[59] R. Miotto, L. Li, B. A. Kidd, and J. T. Dudley, “Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records,” Sci. Rep., vol. 6, no. January, pp. 1–10, 2016, doi: 10.1038/srep26094.
[60] S. J. M. A. E. S. M. R. B. D. R. K. G.-H. S. J. T. S. A. S. F. J. W. M. S. S. A. G. K. S. Aghaeepour, “Author Correction: A machine learning approach to leveraging electronic health records for enhanced omics analysis,” Nat. Mach. Intell., 2025, doi: 10.1038/s42256-025-01021-x.
[61] Z. M. D. H. P. M. Kim, “Multi-omics integration in the age of million single-cell data,” Nat. Rev. Nephrol., 2021, doi: 10.1038/s41581-021-00463-x.
[62] J. Lee et al., “BioBERT: A pre-trained biomedical language representation model for biomedical text mining,” Bioinformatics, vol. 36, no. 4, pp. 1234–1240, 2020, doi: 10.1093/bioinformatics/btz682.
[63] A. Alnattah, M. Jajroudi, S. A. N. Fadafen, M. N. Manzari, and S. Eslami, “Artificial Intelligence in Clinical Decision-Making: A Scoping Review of Rule-Based Systems and Their Applications in Medicine,” Cureus, vol. 17, no. 8, 2025, doi: 10.7759/cureus.91333.
[64] B. Silva, F. Hak, T. Guimaraes, M. Manuel, and M. F. Santos, “Rule-based System for Effective Clinical Decision Support,” Procedia Comput. Sci., vol. 220, no. 2019, pp. 880–885, 2023, doi: 10.1016/j.procs.2023.03.119.
[65] Y. Hasin, M. Seldin, and A. Lusis, “Multi-omics approaches to disease,” Genome Biol., vol. 18, no. 1, pp. 1–15, 2017, doi: 10.1186/s13059-017-1215-1.
[66] V. Sanh et al., “Multitask Prompted Training Enables Zero-Shot Task Generalization,” ICLR 2022 - 10th Int. Conf. Learn. Represent., 2022.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Joshua Machemedze, Belinda Ndlovu
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) ) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
