Object Detection of BISINDO Sign Language Letters Using Residual Network

Abstract

Indonesian Sign Language or BISINDO is an alternative language used by people who suffer from disabilities, especially those who have hearing impairments. This language grew and developed from the deaf community, so its use is based on the visual aspect. This research aims to apply Residual Networks to detect objects in the context of Bisindo Letter Sign Language, with the hope of increasing accuracy and efficiency in letter recognition. Object detection goes through 2 stages, namely feature extraction and model training. ResNet is a type of Convolutional Neural Network (CNN) architecture that utilizes models that have been previously trained, so it can save the time required in the model development process. In this research, Residual Network (ResNet) was used for feature extraction to recognize important aspects in the Bisindo letter sign image, such as hand position, finger shape characteristics, and direction of movement. The research results show that the new dataset used as training data and test data has a fairly good ability to detect with a division of 70% train set, 20% valid set and 10% test set with size 640x640 with 300 epochs for the training model.