Analisis Kontribusi Deteksi Tangan dan Ekstraksi Landmark untuk Pengenalan Alfabet BISINDO


Wa Nanda Sulystrian(1); Muhammad Faisal(2*); Fahrim Irhamna Rachman(3); Abd Rakhim Nanda(4); Rizki Yusliana Bakti(5); Muhammad Syafaat S. Kuba(6); Andi Makbul Syamsuri(7);

(1) Informatika, Universitas Muhammadiyah Makassar, Makassar, Indonesia
(2) Informatika, Universitas Muhammadiyah Makassar, Makassar, Indonesia
(3) Informatika, Universitas Muhammadiyah Makassar, Makassar, Indonesia
(4) Teknik Pengairan, Universitas Muhammadiyah Makassar, Makassar, Indonesia
(5) Informatika, Universitas Muhammadiyah Makassar, Makassar, Indonesia
(6) Teknik Pengairan, Universitas Muhammadiyah Makassar, Makassar, Indonesia
(7) Teknik Pengairan, Universitas Muhammadiyah Makassar, Makassar, Indonesia
(*) Corresponding Author

  

Abstract


Komunitas tunarungu menghadapi hambatan dalam aspek bahasa dan komunikasi sehingga membutuhkan dukungan teknologi yang sesuai dengan karakteristik visual mereka. Kondisi tersebut menunjukkan pentingnya pengembangan teknologi untuk mendukung akses komunikasi yang lebih inklusif melalui pengenalan Bahasa Isyarat Indonesia (BISINDO). Penelitian ini menganalisis kontribusi deteksi tangan dan ekstraksi landmark terhadap performa pengenalan alfabet BISINDO menggunakan pendekatan berbasis computer vision. Sistem yang diusulkan mengintegrasikan YOLOv8 untuk deteksi tangan, MediaPipe Hands untuk ekstraksi dua puluh satu landmark, normalisasi landmark, serta Multi-Layer Perceptron sebagai model klasifikasi. Evaluasi dilakukan menggunakan dataset berisi 1066 citra alfabet BISINDO dari 26 kelas melalui tiga skenario eksperimen. Hasil penelitian menunjukkan bahwa kombinasi deteksi tangan dan normalisasi landmark menghasilkan performa terbaik dengan nilai accuracy 0.915, precision 0.913, recall 0.915, dan F1-score 0.896, serta meningkatkan accuracy 14.8% dibandingkan pendekatan tanpa deteksi tangan. Temuan ini menunjukkan bahwa deteksi tangan dan representasi landmark berkontribusi penting terhadap peningkatan akurasi sistem. Pendekatan yang diusulkan berpotensi diterapkan pada aplikasi penerjemah alfabet BISINDO berbasis kamera secara real-time untuk mendukung komunikasi yang lebih inklusif bagi komunitas tunarungu di Indonesia.

Keywords


BISINDO-Huruf; Deteksi Tangan Berbasis-YOLOv8; MediaPipe Hands; Normalisasi Landmark; Multi-Layer Perceptron

  
  

Full Text:

PDF
  

Article Metrics

Abstract view: 61 times
PDF view: 20 times 		     

Digital Object Identifier

doi  https://doi.org/10.33096/busiti.v7i3.3506 		  

Cite

How to cite item

References


I. Yuniarsih, A. Huda, and U. N. Malang, “Pengembangan Multimedia Interaktif Berbasis Android untuk Meningkatkan Penguasaan Kosakata Siswa Tunarungu,” vol. 8, no. November, pp. 92–97, 2022, doi: 10.17977/um031v8i22022p92-97.

C. Sebastian et al., “Indonesian Sign Language (BISINDO) Recognition Using Spatially Aware Body Gesture Recognition,” Procedia Comput. Sci., vol. 269, pp. 1002–1011, 2025, doi: 10.1016/j.procs.2025.09.042.

R. Jalayer, “Robotics and Computer-Integrated Manufacturing A review on deep learning for vision-based hand detection , hand segmentation and hand gesture recognition in human – robot interaction,” Robot. Comput. Integr. Manuf., vol. 97, no. May 2025, p. 103110, 2026, doi: 10.1016/j.rcim.2025.103110.

M. Perceptron, A. S. Language, A. S. Language, and N. A. Deaf, “Measurement : Sensors 3D geometric features based real-time American sign language recognition using PointNet and MLP with MediaPipe hand skeleton detection,” vol. 38, pp. 2–7, 2025, doi: 10.1016/j.measen.2024.101697.

J. H. Meghana, H. K. Chethan, K. S. S. Kumar, S. P. S. Prakash, and S. P. S. Prakash, “Comprehensive Analysis of Pose Estimation and Machine Learning Classifiers for Precise Yoga Pose Detection and Classification,” Procedia Comput. Sci., vol. 258, pp. 3345–3356, 2025, doi: 10.1016/j.procs.2025.04.592.

X. Chai, M. Zhao, J. Li, and J. Li, “Image small target detection in complex traffic scenes based on Yolov8 multiscale feature fusion,” Alexandria Eng. J., vol. 126, no. May, pp. 578–590, 2025, doi: 10.1016/j.aej.2025.04.105.

D. Lucian, C. Alexander, Y. Raffel, C. Alexander, and Y. Raffel, “Improving Indonesian sign language recognition using lightweight deep learning architectures with knowledge distillation method,” Procedia Comput. Sci., vol. 269, pp. 618–629, 2025, doi: 10.1016/j.procs.2025.09.005.

A. Salem, C. Bang, B. Alsharif, and A. Altaher, “Franklin Open Mamba vision models : Automated American sign language recognition,” Franklin Open, vol. 10, no. March 2024, p. 100224, 2025, doi: 10.1016/j.fraope.2025.100224.

L. A. Ranee, F. L. Marshillong, S. A. Lyndoh, L. Amory, and F. Lyngdoh, “Khasi Sign Language Recognition using Google ’ s Mediapipe and and Deep Learning Feedforward Neural Network Approach,” Procedia Comput. Sci., vol. 258, pp. 3619–3629, 2025, doi: 10.1016/j.procs.2025.04.617.

V. Pradip and P. S. Chakurkar, “MethodsX Towards safer environments : A YOLO and MediaPipe-based human fall detection system with alert automation,” MethodsX, vol. 15, no. September, p. 103623, 2025, doi: 10.1016/j.mex.2025.103623.

J. Badzoka, C. Kappacher, J. Lau, and C. W. Huck, “Accelerating microplastic analysis with machine learning : Utilizing multi-layer perceptron ( MLP ) for classification of hyperspectral data,” vol. 218, no. September, 2025, doi: 10.1016/j.microc.2025.115401.

R. Sreemathy, M. P. Turuk, S. Chaudhary, K. Lavate, A. Ushire, and S. Khurana, “International Journal of Cognitive Computing in Engineering Continuous word level sign language recognition using an expert system based on machine learning,” Int. J. Cogn. Comput. Eng., vol. 4, no. April, pp. 170–178, 2023, doi: 10.1016/j.ijcce.2023.04.002.

R. Raj, R. Sreemathy, M. Turuk, J. Jagdale, and M. Anish, “Performance Comparison of Different Versions of YOLO for Indian Sign Language Captioning in Real Time of Multiple Signers,” Procedia Comput. Sci., vol. 259, pp. 991–1000, 2025, doi: 10.1016/j.procs.2025.04.053.

M. Á. Remiro, M. Gil-Martín, and R. San-Segundo, “Improving Hand Pose Recognition Using Localization and Zoom Normalizations over MediaPipe Landmarks †,” Eng. Proc., vol. 58, no. 1, 2023, doi: 10.3390/ecsa-10-16215.

B. Lagomarsino, A. Massone, F. Odone, M. Casadio, and M. Moro, “Video-based markerless assessment of bilateral upper limb motor activity following cervical spinal cord injury,” Comput. Biol. Med., vol. 196, no. PC, p. 110908, 2025, doi: 10.1016/j.compbiomed.2025.110908.

T. Ahmed, J. Tulsi, M. Alam, K. Kadir, K. Mansoor, and M. S. Mazliham, “Analysis and visualization of fraud detection patterns through data mining and classification using MLP and hybrid deep learning model,” Egypt. Informatics J., vol. 32, no. May, p. 100829, 2025, doi: 10.1016/j.eij.2025.100829.


Refbacks

  • There are currently no refbacks.


Copyright (c) 2026 Buletin Sistem Informasi dan Teknologi Islam (BUSITI)

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.