Japanese Sign Language Recognition by Combining Joint Skeleton-Based Handcrafted and Pixel-Based Deep Learning Features with Machine Learning Classification

Sign language recognition is vital for enhancing communication accessibility among the Deaf and hard-of-hearing communities.In Japan,approximately 360,000 individualswith hearing and speech disabilities rely on Japanese Sign Language(JSL)for communication.However,existing JSL recognition systems have faced significant performance limitations due to inherent complexities.In response to these challenges,we present a novel JSL recognition system that employs a strategic fusion approach,combining joint skeleton-based handcrafted features and pixel-based deep learning features.Our system incorporates two distinct streams:the first stream extracts crucial handcrafted features,emphasizing the capture of hand and body movements within JSL gestures.Simultaneously,a deep learning-based transfer learning stream captures hierarchical representations of JSL gestures in the second stream.Then,we concatenated the critical information of the first stream and the hierarchy of the second stream features to produce the multiple levels of the fusion features,aiming to create a comprehensive representation of the JSL gestures.After reducing the dimensionality of the feature,a feature selection approach and a kernel-based support vector machine(SVM)were used for the classification.To assess the effectiveness of our approach,we conducted extensive experiments on our Lab JSL dataset and a publicly available Arabic sign language(ArSL)dataset.Our results unequivocally demonstrate that our fusion approach significantly enhances JSL recognition accuracy and robustness compared to individual feature sets or traditional recognition methods.

Paired regions for shadow removal approach based on multi-features

The existence of shadow leads to the degradation of the image qualities and the defect of ground object information.Shadow removal is therefore an essential research topic in image processing filed.The biggest challenge of shadow removal is how to restore the content of shadow areas correctly while removing the shadow in the image.Paired regions for shadow removal approach based on multi-features is proposed, in which shadow removal is only performed on related sunlit areas.Feature distance between regions is calculated to find the optimal paired regions with considering of multi-features(texture, gradient feature, etc.) comprehensively.Images in different scenes with peak signal-to-noise ratio(PSNR) and structural similarity(SSIM) evaluation indexes are chosen for experiments.The results are shown with six existing comparison methods by visual and quantitative assessments, which verified that the proposed method shows excellent shadow removal effect, the brightness, color of the removed shadow area, and the surrounding non-shadow area can be naturally fused.

A Multiple Feature Approach for Disorder Normalization in Clinical Notes

In this paper we propose a multiple feature approach for the normalization task which can map each disorder mention in the text to a unique unified medical language system（UMLS）concept unique identifier（CUI）. We develop a two-step method to acquire a list of candidate CUIs and their associated preferred names using UMLS API and to choose the closest CUI by calculating the similarity between the input disorder mention and each candidate. The similarity calculation step is formulated as a classification problem and multiple features（string features,ranking features,similarity features,and contextual features） are used to normalize the disorder mentions. The results show that the multiple feature approach improves the accuracy of the normalization task from 32.99% to 67.08% compared with the Meta Map baseline.