Cross-media retrieval is an interesting research topic,which seeks to remove the barriers among different modalities.To enable cross-media retrieval,it is needed to find the correlation measures between heterogeneous ...Cross-media retrieval is an interesting research topic,which seeks to remove the barriers among different modalities.To enable cross-media retrieval,it is needed to find the correlation measures between heterogeneous low-level features and to judge the semantic similarity.This paper presents a novel approach to learn cross-media correlation between visual features and auditory features for image-audio retrieval.A semi-supervised correlation preserving mapping(SSCPM)method is described to construct the isomorphic SSCPM subspace where canonical correlations between the original visual and auditory features are further preserved.Subspace optimization algorithm is proposed to improve the local image cluster and audio cluster quality in an interactive way.A unique relevance feedback strategy is developed to update the knowledge of cross-media correlation by learning from user behaviors,so retrieval performance is enhanced in a progressive manner.Experimental results show that the performance of our approach is effective.展开更多
This paper presents a cross-media semantic mining model （CSMM） based on object semantic. This model obtains object-level semantic information in terms of maximum probability principle. Then semantic templates are tr...This paper presents a cross-media semantic mining model （CSMM） based on object semantic. This model obtains object-level semantic information in terms of maximum probability principle. Then semantic templates are trained and constructed with STTS （Semantic Template Training System）, which are taken as the bridge to realize the transition from various low-level media feature to object semantic. Furthermore, we put forward a kind of double layers metadata structure to efficaciously store and manage mined low-level feature and high-level semantic. This model has broad application in lots of domains such as intelligent retrieval engine, medical diagnoses, multimedia design and so on.展开更多
基金Project supported by the National Natural Science Foundation of China (Nos. 60533090 and 60773051)the Natural Science Foundation of Zhejiang Province (No. Y105395),China
文摘Cross-media retrieval is an interesting research topic,which seeks to remove the barriers among different modalities.To enable cross-media retrieval,it is needed to find the correlation measures between heterogeneous low-level features and to judge the semantic similarity.This paper presents a novel approach to learn cross-media correlation between visual features and auditory features for image-audio retrieval.A semi-supervised correlation preserving mapping(SSCPM)method is described to construct the isomorphic SSCPM subspace where canonical correlations between the original visual and auditory features are further preserved.Subspace optimization algorithm is proposed to improve the local image cluster and audio cluster quality in an interactive way.A unique relevance feedback strategy is developed to update the knowledge of cross-media correlation by learning from user behaviors,so retrieval performance is enhanced in a progressive manner.Experimental results show that the performance of our approach is effective.
基金Supported by the National Basic Research Program of China 973 Program (2007CB310801)the Specialized Research Fund for the Doctoral Program of Higer Education of China (20070486064)+1 种基金the Natural Science Foundation of Hubei Province (2007ABA038)the Programme of Introducing Talents of Discipline to Universities (B07037)
文摘This paper presents a cross-media semantic mining model （CSMM） based on object semantic. This model obtains object-level semantic information in terms of maximum probability principle. Then semantic templates are trained and constructed with STTS （Semantic Template Training System）, which are taken as the bridge to realize the transition from various low-level media feature to object semantic. Furthermore, we put forward a kind of double layers metadata structure to efficaciously store and manage mined low-level feature and high-level semantic. This model has broad application in lots of domains such as intelligent retrieval engine, medical diagnoses, multimedia design and so on.
