Named Entity Recognition for Medical Records of Heart Failure Using a Pre-trained BERT Model
DOI:
https://doi.org/10.30871/jaic.v9i2.9170Keywords:
Named Entity Recognition, BERT, Medical Records, Heart Failure, Transformer, Medical Text ClassificationAbstract
This study aims to develop a Named Entity Recognition (NER) model based on a pre-trained BERT model for medical records of heart failure patients. The focus of this research is to classify essential medical entities from unstructured medical record texts. The classification covers four categories: objective data (patient identity, laboratory test results, and objective examination data), subjective data (patient complaints), prescriptions, and diagnoses (diagnosis codes and descriptions). The methodology employs Natural Language Processing (NLP) techniques using Transformer-based architectures, such as Bidirectional Encoder Representation from Transformers (BERT). The developed model is evaluated based on entity label prediction accuracy and medical entity classification performance. The results indicate that the BERT-based NER model performs well, achieving an entity prediction accuracy of 84.82%. Furthermore, the model effectively classifies medical entities from input texts in alignment with expected medical entities. This research is expected to contribute significantly to medical data management, assist healthcare professionals in clinical decision-making, and serve as a reference for the development of AI-based healthcare technology in Indonesia.
Downloads
References
[1] A. A. Morris, “Looking North to GUIDE Better Care for Heart Failure Is Not Black or White,” JACC: Heart Failure.
[2] J. Rangaswami and P. A. McCullough, “Clinical Context of Dyskalemias Across the Heart Failure Spectrum and Their Associated Adverse Outcomes,” JACC Heart Fail, vol. 7, no. 6, p. 533, Jun. 2019, doi: 10.1016/j.jchf.2019.01.005.
[3] D. Jurafsky and J. H. Martin, “Speech and Language Processing An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models Third Edition draft Summary of Contents.”
[4] E. H. Houssein, R. E. Mohamed, G. Hu, and A. A. Ali, “Adapting transformer-based language models for heart disease detection and risk factors extraction,” J Big Data, vol. 11, no. 1, Dec. 2024, doi: 10.1186/s40537-024-00903-y.
[5] A. Vaswani et al., “Attention Is All You Need,” Jun. 2017, [Online]. Available: http://arxiv.org/abs/1706.03762
[6] C. Wang et al., “Named entity recognition (NER) for Chinese agricultural diseases and pests based on discourse topic and attention mechanism,” Evol Intell, vol. 17, no. 1, pp. 457–466, Feb. 2024, doi: 10.1007/s12065-022-00727-w.
[7] H. Pooja and M. P. P. Jagadeesh, “A Deep Learning Based Approach for Biomedical Named Entity Recognition Using Multitasking Transfer Learning with BiLSTM, BERT and CRF,” SN Comput Sci, vol. 5, no. 5, Jun. 2024, doi: 10.1007/s42979-024-02835-z.
[8] A. Vaswani et al., “Attention Is All You Need,” Jun. 2017, [Online]. Available: http://arxiv.org/abs/1706.03762
[9] P. Chen, M. Zhang, X. Yu, and S. Li, “Named entity recognition of Chinese electronic medical records based on a hybrid neural network and medical MC-BERT,” BMC Med Inform Decis Mak, vol. 22, no. 1, Dec. 2022, doi: 10.1186/s12911-022-02059-2.
[10] A. Thukral, S. Dhiman, R. Meher, and P. Bedi, “Knowledge graph enrichment from clinical narratives using NLP, NER, and biomedical ontologies for healthcare applications,” International Journal of Information Technology (Singapore), vol. 15, no. 1, pp. 53–65, Jan. 2023, doi: 10.1007/s41870-022-01145-y.
[11] Á. García-Barragán, A. González Calatayud, O. Solarte-Pabón, M. Provencio, E. Menasalvas, and V. Robles, “GPT for medical entity recognition in Spanish,” Multimed Tools Appl, 2024, doi: 10.1007/s11042-024-19209-5.
[12] R. P. Kusumawardani and K. N. Kusumawati, “Named entity recognition in the medical domain for Indonesian language health consultation services using bidirectional-lstmcrf algorithm,” in Procedia Computer Science, Elsevier B.V., 2024, pp. 1146–1156. doi: 10.1016/j.procs.2024.10.344
[13] N. Liu, Q. Hu, H. Xu, X. Xu and M. Chen, "Med-BERT: A Pretraining Framework for Medical Records Named Entity Recognition," in IEEE Transactions on Industrial Informatics, vol. 18, no. 8, pp. 5600-5608, Aug. 2022, doi: 10.1109/TII.2021.3131180.
[14] U. Naseem, M. Khushi, V. Reddy, S. Rajendran, I. Razzak and J. Kim, "BioALBERT: A Simple and Effective Pre-trained Language Model for Biomedical Named Entity Recognition," 2021 International Joint Conference on Neural Networks (IJCNN), Shenzhen, China, 2021, pp. 1-7, doi: 10.1109/IJCNN52387.2021.9533884.
[15] M. U. Javeed, M. S. Ali, A. Iqbal, M. Azhar, S. M. Aslam and I. Shabbir, "Transforming Heart Disease Detection with BERT: Novel Architectures and Fine-Tuning Techniques," 2024 International Conference on Frontiers of Information Technology (FIT), Islamabad, Pakistan, 2024, pp. 1-6, doi: 10.1109/FIT63703.2024.10838424.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Mikael Triartama Manurung, I Gusti Ngurah Lanang Wijayakusuma, I Putu Winada Gautama
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) ) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
