Background Gesture recognition has attracted significant attention because of its wide range of potential applications.Although multi-modal gesture recognition has made significant progress in recent years,a popular m...Background Gesture recognition has attracted significant attention because of its wide range of potential applications.Although multi-modal gesture recognition has made significant progress in recent years,a popular method still is simply fusing prediction scores at the end of each branch,which often ignores complementary features among different modalities in the early stage and does not fuse the complementary features into a more discriminative feature.Methods This paper proposes an Adaptive Cross-modal Weighting(ACmW)scheme to exploit complementarity features from RGB-D data in this study.The scheme learns relations among different modalities by combining the features of different data streams.The proposed ACmW module contains two key functions:(1)fusing complementary features from multiple streams through an adaptive one-dimensional convolution;and(2)modeling the correlation of multi-stream complementary features in the time dimension.Through the effective combination of these two functional modules,the proposed ACmW can automatically analyze the relationship between the complementary features from different streams,and can fuse them in the spatial and temporal dimensions.Results Extensive experiments validate the effectiveness of the proposed method,and show that our method outperforms state-of-the-art methods on IsoGD and NVGesture.展开更多
Dear Editor,Bread wheat(Triticum aestivum L.)is a globally important cereal providing~20% of the calories and proteins for>4.5billion people.Plant architecture,including morphologies of leaves,spikes,stems,and root...Dear Editor,Bread wheat(Triticum aestivum L.)is a globally important cereal providing~20% of the calories and proteins for>4.5billion people.Plant architecture,including morphologies of leaves,spikes,stems,and roots,has great impact on plant development and productivity,and thus has been extensively investigated in various plant species(Jiang et al.,2023;Zhang et al.,2017).展开更多
基金the Chinese National Natural Science Foundation Projects(61961160704,61876179)the Key Project of the General Logistics Department(ASW17C001)the Science and Technology Development Fund of Macao(0010/2019/AFJ,0025/2019/AKP).
文摘Background Gesture recognition has attracted significant attention because of its wide range of potential applications.Although multi-modal gesture recognition has made significant progress in recent years,a popular method still is simply fusing prediction scores at the end of each branch,which often ignores complementary features among different modalities in the early stage and does not fuse the complementary features into a more discriminative feature.Methods This paper proposes an Adaptive Cross-modal Weighting(ACmW)scheme to exploit complementarity features from RGB-D data in this study.The scheme learns relations among different modalities by combining the features of different data streams.The proposed ACmW module contains two key functions:(1)fusing complementary features from multiple streams through an adaptive one-dimensional convolution;and(2)modeling the correlation of multi-stream complementary features in the time dimension.Through the effective combination of these two functional modules,the proposed ACmW can automatically analyze the relationship between the complementary features from different streams,and can fuse them in the spatial and temporal dimensions.Results Extensive experiments validate the effectiveness of the proposed method,and show that our method outperforms state-of-the-art methods on IsoGD and NVGesture.
基金supported by the Key R&D Program of Shandong Province(ZR202211070163,2023LZGC022)the Provincial Natural Science Foundation of Shandong(ZR2021MC056,ZR2021ZD30)+1 种基金the National Key Research and Development Program of China(2022YFD1201300)the Young Taishan Scholars Program of Shandong Province。
文摘Dear Editor,Bread wheat(Triticum aestivum L.)is a globally important cereal providing~20% of the calories and proteins for>4.5billion people.Plant architecture,including morphologies of leaves,spikes,stems,and roots,has great impact on plant development and productivity,and thus has been extensively investigated in various plant species(Jiang et al.,2023;Zhang et al.,2017).