Development of a Vision-Based Hand Movement Detection System for Post-Stroke Rehabilitation
DOI:
https://doi.org/10.30871/ji.v18i1.11414Keywords:
YOLOv8, Jetson Xavier NX, Hand Gesture, Computer Vision, ESP32Abstract
The need for effective and affordable post-stroke rehabilitation technology has driven the development of a finger gesture detection system based on YOLOv8 implemented on the Jetson Xavier NX. The system is designed to recognize hand gestures—including open hand, closed hand, and thumb touching other fingers—in real time using a camera as the image input. The detection results are then transmitted to an ESP32 microcontroller to control LEDs as visual indicators. The research process involves hardware and software design, model training using datasets from Roboflow, and system testing on the Jetson Xavier NX. Experimental results show that the system can detect gestures with high accuracy and fast response. These findings demonstrate the potential of computer vision based on edge computing as a foundation for assistive technologies in both medical rehabilitation and human–computer interaction.
Downloads
References
[1] M. A. Fakhruddin, H. Pratikno, Musayyanah, and W. I. Kusumawati, “Kontrol Level Kecepatan Kipas Melalui Deteksi Gestur Jari Tangan Menggunakan MediaPipe dan Faster-RCNN,” Jurnal Teknologi Informasi dan Ilmu Komputer, vol. 10, no. 6, pp. 1295–1302, Dec. 2023, doi: 10.25126/jtiik.2023107345.
[2] S. Ka, I. Habriansyah, A. Pailan, A. Muh Rizky Ramadadhan, and D. Jurusan, “ROBOT REHABILITASI LENGAN BAGI PENDERITA STROKE.”
[3] J. Pransisko, R. Ardeanto, and A. Ramandani, “LITERATUR REVIEW SISTEM CERDAS IMAGE PROCESSING DAN KLASIFIKASI CITRA MENGGUNAKAN CONVOLUTIONAL NEURAL NETWORK (CNN).”[Online].Available: https://www.researchgate.net/publication/351356846
[4] J. Jumadi, Y. Yupianti, and D. Sartika, “PENGOLAHAN CITRA DIGITAL UNTUK IDENTIFIKASI OBJEK MENGGUNAKAN METODE HIERARCHICAL AGGLOMERATIVE CLUSTERING,” JST (Jurnal Sains dan Teknologi), vol. 10, no. 2, pp. 148–156, Nov. 2021, doi: 10.23887/jst-undiksha.v10i2.33636.
[5] J. Subur et al., “CYCLOTRON : Jurnal Teknik Elektro Pemanfaatan Teknologi Computer Vision untuk Deteksi Ukuran Ikan Bandeng dalam Membantu Proses Sortir Ikan”.
[6] M. Abdul muthalib, I. Irfan, K. Kartika, and S. M. Selamat Meliala, “PENGIRAAN POSE MODEL MANUSIA PADA REPETISI KEBUGARAN AI PEMOGRAMAN PYTHON BERBASIS KOMPUTERISASI,” INFOTECH journal, vol. 9, no. 1, pp. 11–19, Jan. 2023, doi: 10.31949/infotech.v9i1.4233.
[7] A. Sani and S. Rahmadinni, “Deteksi Gestur Tangan Berbasis Pengolahan Citra,” Jurnal Rekayasa Elektrika, vol. 18, no. 2, Jul. 2022, doi: 10.17529/jre.v18i2.25147.
[8] MUH. IKBAL and R. A. Saputra, “PENGENALAN RAMBU LALU LINTAS MENGGUNAKAN METODE YOLOV8,” JIKA (Jurnal Informatika), vol. 8, no. 2, p. 204, Apr. 2024, doi: 10.31000/jika.v8i2.10609.
[9] A. M. Chalik, A. Qowy, F. Hanafi, and A. Nuraminah, “Mouse Tracking Tangan dengan Klasifikasi Gestur Menggunakan OpenCV dan Mediapipe,” JUITIK, vol. 1, no. 2, 2021, [Online].Available: http://journal.sinov.id/index.php/juitik/indexHalamanUTAMAJurnal:https://journal.sinov.id/index.php
[10] “Perbandingan Performa Jetson Nano, Jetson Xavier NX dan Lenovo Legion 5 terhadap Penggunaan YOLOv7.”
[11] J. Ali, “SISTEM SECURITY WEBCAM DENGAN MENGGUNAKAN MICROSOFT VISUAL BASIC (6.0),” Jurnal Teknologi dan Sistem Informasi UNIVRAB, vol. 1, no. 2, 2016.
[12] F. M. Syakir and M. Syani, “APLIKASI TAMU WAJIB LAPOR BERBASIS MOBILE (STUDI KASUS KP. PASIR PEUNDEUY CIHAMPELAS BANDUNG BARAT),” Jurnal Informatika dan Teknik Elektro Terapan, vol. 12, no. 3S1, Oct. 2024, doi: 10.23960/jitet.v12i3s1.5211.
[13] N. Sujana, M. Malik Mutoffar, and A. Azzam Haryanto, “ANALISIS KINERJA YOLOV8 OPTIMALISASI ROBOFLOW UNTUK DETEKSI EKSPRESI WAJAH EMOSIONAL DENGAN MACHINE LEARNING,” vol. 06, 2024.
[14] Z. B. Abilovani, W. Yahya, and F. A. Bakhtiar, “Implementasi Protokol MQTT Untuk Sistem Monitoring Perangkat IoT,” 2018. [Online]. Available: http://j-ptiik.ub.ac.id
[15] G. Yudha Saputra, A. Denhas Afrizal, F. Khusnu Reza Mahfud, F. Angga Pribadi, and F. Jati Pamungkas, “PENERAPAN PROTOKOL MQTT PADA TEKNOLOGI WAN (STUDI KASUS SISTEM PARKIR UNIVERISTAS BRAWIJAYA),” 2017.
[16] C. Putra Hadisusila, “Aplikasi Arduino dalam Teknik I/O untuk Mengintegrasikan dan Mengendalikan Perangkat Elektronik,” Jurnal Nusantara Of Engineering, vol. 6, no. 2, 2023, [Online].Available: https://ojs.unpkediri.ac.id/index.php/noe
[17] A. Pratikto Wahyu Hendrawan, N. Putu Agustini Teknik Elektro ITN Malang, K.-K. Km, and K. Kunci, “Simulasi Kendali Dan Monitoring Daya Listrik Peralatan Rumah Tangga Berbasis ESP32.”[Online].Available: www.elektro.itn.ac.id
[18] A. G V, N. T, M. K S, S. S, and Dr. A. M, “Deep learning- driven myoelectric gesture classification for post-stroke rehabilitation,” IJARCCE, vol. 14, no. 11, Nov. 2025, doi: 10.17148/ijarcce.2025.141178.
[19] H. Badi, “Recent methods in vision-based hand gesture recognition,” Int. J. Data Sci. Anal., vol. 1, no. 2, pp. 77–87, Jul. 2016, doi: 10.1007/s41060-016-0008-z.
[20] S. Karthik, S. Bhaggiaraj, E. S. Soji, L. S. Deve, S. S. Priscila, and S. S. Rajest, “AVE Trends in Intelligent Computing Systems Real-Time Hand Gesture Recognition for Stroke Rehabilitation Using Deep Learning Approach,” 2024.
[21] W. H. Chang and Y.-H. Kim, “Robot-assisted Therapy in Stroke Rehabilitation,” J. Stroke, vol. 15, no. 3, p. 174, 2013, doi: 10.5853/jos.2013.15.3.174.
[22] S. N. Sidek, A. U. Shamsudin, and E. Ismail, “A hybrid controller with Chedoke-McMaster stroke assessment for robot-assisted rehabilitation,” in Procedia Engineering, Elsevier Ltd, 2012, pp. 629–635. doi: 10.1016/j.proeng.2012.07.222.
