Alphabet-Level Indian Sign Language Translation to Text Using Hybrid-AO Thresholding with CNN

Sign language is used as a communication medium in the field of trade,defence,and in deaf-mute communities worldwide.Over the last few decades,research in the domain of translation of sign language has grown and become more challenging.This necessitates the development of a Sign Language Translation System(SLTS)to provide effective communication in different research domains.In this paper,novel Hybrid Adaptive Gaussian Thresholding with Otsu Algorithm(Hybrid-AO)for image segmentation is proposed for the translation of alphabet-level Indian Sign Language(ISLTS)with a 5-layer Convolution Neural Network(CNN).The focus of this paper is to analyze various image segmentation(Canny Edge Detection,Simple Thresholding,and Hybrid-AO),pooling approaches(Max,Average,and Global Average Pooling),and activation functions(ReLU,Leaky ReLU,and ELU).5-layer CNN with Max pooling,Leaky ReLU activation function,and Hybrid-AO(5MXLR-HAO)have outperformed other frameworks.An open-access dataset of ISL alphabets with approx.31 K images of 26 classes have been used to train and test the model.The proposed framework has been developed for translating alphabet-level Indian Sign Language into text.The proposed framework attains 98.95%training accuracy,98.05%validation accuracy,and 0.0721 training loss and 0.1021 validation loss and the perfor-mance of the proposed system outperforms other existing systems.

An approach based on deep learning for Indian sign language translation

Purpose–According to the Indian Sign Language Research and Training Centre(ISLRTC),India has approximately 300 certified human interpreters to help people with hearing loss.This paper aims to address the issue of Indian Sign Language(ISL)sentence recognition and translation into semantically equivalent English text in a signer-independent mode.Design/methodology/approach–This study presents an approach that translates ISL sentences into English text using the MobileNetV2 model and Neural Machine Translation(NMT).The authors have created an ISL corpus from the Brown corpus using ISL grammar rules to perform machine translation.The authors’approach converts ISL videos of the newly created dataset into ISL gloss sequences using the MobileNetV2 model and the recognized ISL gloss sequence is then fed to a machine translation module that generates an English sentence for each ISL sentence.Findings–As per the experimental results,pretrained MobileNetV2 model was proven the best-suited model for the recognition of ISL sentences and NMT provided better results than Statistical Machine Translation(SMT)to convert ISL text into English text.The automatic and human evaluation of the proposed approach yielded accuracies of 83.3 and 86.1%,respectively.Research limitations/implications–It can be seen that the neural machine translation systems produced translations with repetitions of other translated words,strange translations when the total number of words per sentence is increased and one or more unexpected terms that had no relation to the source text on occasion.The most common type of error is the mistranslation of places,numbers and dates.Although this has little effect on the overall structure of the translated sentence,it indicates that the embedding learned for these few words could be improved.Originality/value–Sign language recognition and translation is a crucial step toward improving communication between the deaf and the rest of society.Because of the shortage of human interpreters,an alternative approach is desired to help people achieve smooth communication with the Deaf.To motivate research in this field,the authors generated an ISL corpus of 13,720 sentences and a video dataset of 47,880 ISL videos.As there is no public dataset available for ISl videos incorporating signs released by ISLRTC,the authors created a new video dataset and ISL corpus.

A Robust Model for Translating Arabic Sign Language into Spoken Arabic Using Deep Learning

This study presents a novel and innovative approach to auto-matically translating Arabic Sign Language(ATSL)into spoken Arabic.The proposed solution utilizes a deep learning-based classification approach and the transfer learning technique to retrain 12 image recognition models.The image-based translation method maps sign language gestures to corre-sponding letters or words using distance measures and classification as a machine learning technique.The results show that the proposed model is more accurate and faster than traditional image-based models in classifying Arabic-language signs,with a translation accuracy of 93.7%.This research makes a significant contribution to the field of ATSL.It offers a practical solution for improving communication for individuals with special needs,such as the deaf and mute community.This work demonstrates the potential of deep learning techniques in translating sign language into natural language and highlights the importance of ATSL in facilitating communication for individuals with disabilities.