Computer vision-based six layered ConvNeural network to recognize sign language for both numeral and alphabet signs

People who have trouble communicating verbally are often dependent on sign language,which can be difficult for most people to understand,making interaction with them a difficult endeavor.The Sign Language Recognition(SLR)system takes an input expression from a hearing or speaking-impaired person and outputs it in the form of text or voice to a normal person.The existing study related to the Sign Language Recognition system has some drawbacks,such as a lack of large datasets and datasets with a range of backgrounds,skin tones,and ages.This research efficiently focuses on Sign Language Recognition to overcome previous limitations.Most importantly,we use our proposed Convolutional Neural Network(CNN)model,“ConvNeural”,in order to train our dataset.Additionally,we develop our own datasets,“BdSL_OPSA22_STATIC1”and“BdSL_OPSA22_STATIC2”,both of which have ambiguous backgrounds.“BdSL_OPSA22_STATIC1”and“BdSL_OPSA22_STATIC2”both include images of Bangla characters and numerals,a total of 24,615 and 8437 images,respectively.The“ConvNeural”model outperforms the pre-trained models with accuracy of 98.38%for“BdSL_OPSA22_STATIC1”and 92.78%for“BdSL_OPSA22_STATIC2”.For“BdSL_OPSA22_STATIC1”dataset,we get precision,recall,F1-score,sensitivity and specificity of 96%,95%,95%,99.31%,and 95.78%respectively.Moreover,in case of“BdSL_OPSA22_STATIC2”dataset,we achieve precision,recall,F1-score,sensitivity and specificity of 90%,88%,88%,100%,and 100%respectively.

Bangla language modeling algorithm for automatic recognition of hand-sign-spelled Bangla sign language

Because of using traditional hand-sign segmentation and classification algorithm,many diversities of Bangla language including joint-letters,dependent vowels etc.and representing 51 Bangla written characters by using only 36 hand-signs,continuous hand-sign-spelled Bangla sign language(BdSL)recognition is challenging.This paper presents a Bangla language modeling algorithm for automatic recognition of hand-sign-spelled Bangla sign language which consists of two phases.First phase is designed for hand-sign classification and the second phase is designed for Bangla language modeling algorithm(BLMA)for automatic recognition of hand-sign-spelled Bangla sign language.In first phase,we have proposed two step classifiers for hand-sign classification using normalized outer boundary vector(NOBV)and window-grid vector(WGV)by calculating maximum inter correlation coefficient(ICC)between test feature vector and pre-trained feature vectors.At first,the system classifies hand-signs using NOBV.If classification score does not satisfy specific threshold then another classifier based on WGV is used.The system is trained using 5,200 images and tested using another(5,200×6)images of 52 hand-signs from 10 signers in 6 different challenging environments achieving mean accuracy of 95.83%for classification with the computational cost of 39.972 milliseconds per frame.In the Second Phase,we have proposed Bangla language modeling algorithm(BLMA)which discovers all"hidden characters"based on"recognized characters"from 52 hand-signs of BdSL to make any Bangla words,composite numerals and sentences in BdSL with no training,only based on the result of first phase.To the best of our knowledge,the proposed system is the first system in BdSL designed on automatic recognition of hand-sign-spelled BdSL for large lexicon.The system is tested for BLMA using hand-sign-spelled 500 words,100 composite numerals and 80 sentences in BdSL achieving mean accuracy of 93.50%,95.50%and 90.50%respectively.