The improved scene-based adaptive nonuniformity correction (NUC) algorithms using a neural network (NNT) approach for infrared image sequences are presented and analyzed. The retina-like neural networks using steepest...The improved scene-based adaptive nonuniformity correction (NUC) algorithms using a neural network (NNT) approach for infrared image sequences are presented and analyzed. The retina-like neural networks using steepest descent model was the first proposed infrared focal plane arrays (IRFPA) nonuniformity compensation method,which can perform parameter estimation of the sensors over time on a frame by frame basis. To increase the strength and the robustness of the NNT algorithm and to avoid the presence of ghosting artifacts,some optimization techniques,including momentum term,regularization factor and adaptive learning rate,were executed in the parameter learning process. In this paper,the local median filtering result of AX^U_ ij (n) is proposed as an alternative value of desired network output of neuron X_ ij (n),denoted as T_ ij (n),which is the local spatial average of AX^U_ ij (n) in traditional NNT methods. Noticeably,the NUC algorithm is inter-frame adaptive in nature and does not rely on any statistical assumptions on the scene data in the image sequence. Applications of this algorithm to the simulated video sequences and real infrared data taken with PV320 show that the correction results of image sequence are better than that of using original NNT approach,especially for the short-time image sequences (several hundred frames) subjected to the dense impulse noises with a number of dead or saturated pixels.展开更多
Considering the relatively poor robustness of quality scores for different types of distortion and the lack of mechanism for determining distortion types, a no-reference image quality assessment(NR-IQA) method based o...Considering the relatively poor robustness of quality scores for different types of distortion and the lack of mechanism for determining distortion types, a no-reference image quality assessment(NR-IQA) method based on the Ada Boost BP neural network in the wavelet domain(WABNN) is proposed. A 36-dimensional image feature vector is constructed by extracting natural scene statistics(NSS) features and local information entropy features of the distorted image wavelet sub-band coefficients in three scales. The ABNN classifier is obtained by learning the relationship between image features and distortion types. The ABNN scorer is obtained by learning the relationship between image features and image quality scores. A series of contrast experiments are carried out in the laboratory of image and video engineering(LIVE) database and TID2013 database. Experimental results show the high accuracy of the distinguishing distortion type, the high consistency with subjective scores and the high robustness of the method for distorted images. Experiment results also show the independence of the database and the relatively high operation efficiency of this method.展开更多
Spectrogram representations of acoustic scenes have achieved competitive performance for acoustic scene classification. Yet, the spectrogram alone does not take into account a substantial amount of time-frequency info...Spectrogram representations of acoustic scenes have achieved competitive performance for acoustic scene classification. Yet, the spectrogram alone does not take into account a substantial amount of time-frequency information. In this study, we present an approach for exploring the benefits of deep scalogram representations, extracted in segments from an audio stream. The approach presented firstly transforms the segmented acoustic scenes into bump and morse scalograms, as well as spectrograms; secondly, the spectrograms or scalograms are sent into pre-trained convolutional neural networks; thirdly,the features extracted from a subsequent fully connected layer are fed into(bidirectional) gated recurrent neural networks, which are followed by a single highway layer and a softmax layer;finally, predictions from these three systems are fused by a margin sampling value strategy. We then evaluate the proposed approach using the acoustic scene classification data set of 2017 IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events(DCASE). On the evaluation set, an accuracy of 64.0 % from bidirectional gated recurrent neural networks is obtained when fusing the spectrogram and the bump scalogram, which is an improvement on the 61.0 % baseline result provided by the DCASE 2017 organisers. This result shows that extracted bump scalograms are capable of improving the classification accuracy,when fusing with a spectrogram-based system.展开更多
Augmented reality is the merging of synthetic sensory information into a user's perception of a real environment. As one of the most important tasks in augmented scene modeling, terrain simplification research has...Augmented reality is the merging of synthetic sensory information into a user's perception of a real environment. As one of the most important tasks in augmented scene modeling, terrain simplification research has gained more and more attention. In this paper, we mainly focus on point selection problem in terrain simplification using triangulated irregular network. Based on the analysis and comparison of traditional importance measures for each input point, we put forward a new importance measure based on local entropy. The results demonstrate that the local entropy criterion has a better performance than any traditional methods. In addition, it can effectively conquer the 'short-sight' problem associated with the traditional methods.展开更多
文摘The improved scene-based adaptive nonuniformity correction (NUC) algorithms using a neural network (NNT) approach for infrared image sequences are presented and analyzed. The retina-like neural networks using steepest descent model was the first proposed infrared focal plane arrays (IRFPA) nonuniformity compensation method,which can perform parameter estimation of the sensors over time on a frame by frame basis. To increase the strength and the robustness of the NNT algorithm and to avoid the presence of ghosting artifacts,some optimization techniques,including momentum term,regularization factor and adaptive learning rate,were executed in the parameter learning process. In this paper,the local median filtering result of AX^U_ ij (n) is proposed as an alternative value of desired network output of neuron X_ ij (n),denoted as T_ ij (n),which is the local spatial average of AX^U_ ij (n) in traditional NNT methods. Noticeably,the NUC algorithm is inter-frame adaptive in nature and does not rely on any statistical assumptions on the scene data in the image sequence. Applications of this algorithm to the simulated video sequences and real infrared data taken with PV320 show that the correction results of image sequence are better than that of using original NNT approach,especially for the short-time image sequences (several hundred frames) subjected to the dense impulse noises with a number of dead or saturated pixels.
基金supported by the National Natural Science Foundation of China(61471194 61705104)+1 种基金the Science and Technology on Avionics Integration Laboratory and Aeronautical Science Foundation of China(20155552050)the Natural Science Foundation of Jiangsu Province(BK20170804)
文摘Considering the relatively poor robustness of quality scores for different types of distortion and the lack of mechanism for determining distortion types, a no-reference image quality assessment(NR-IQA) method based on the Ada Boost BP neural network in the wavelet domain(WABNN) is proposed. A 36-dimensional image feature vector is constructed by extracting natural scene statistics(NSS) features and local information entropy features of the distorted image wavelet sub-band coefficients in three scales. The ABNN classifier is obtained by learning the relationship between image features and distortion types. The ABNN scorer is obtained by learning the relationship between image features and image quality scores. A series of contrast experiments are carried out in the laboratory of image and video engineering(LIVE) database and TID2013 database. Experimental results show the high accuracy of the distinguishing distortion type, the high consistency with subjective scores and the high robustness of the method for distorted images. Experiment results also show the independence of the database and the relatively high operation efficiency of this method.
基金supported by the German National BMBF IKT2020-Grant(16SV7213)(EmotAsS)the European-Unions Horizon 2020 Research and Innovation Programme(688835)(DE-ENIGMA)the China Scholarship Council(CSC)
文摘Spectrogram representations of acoustic scenes have achieved competitive performance for acoustic scene classification. Yet, the spectrogram alone does not take into account a substantial amount of time-frequency information. In this study, we present an approach for exploring the benefits of deep scalogram representations, extracted in segments from an audio stream. The approach presented firstly transforms the segmented acoustic scenes into bump and morse scalograms, as well as spectrograms; secondly, the spectrograms or scalograms are sent into pre-trained convolutional neural networks; thirdly,the features extracted from a subsequent fully connected layer are fed into(bidirectional) gated recurrent neural networks, which are followed by a single highway layer and a softmax layer;finally, predictions from these three systems are fused by a margin sampling value strategy. We then evaluate the proposed approach using the acoustic scene classification data set of 2017 IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events(DCASE). On the evaluation set, an accuracy of 64.0 % from bidirectional gated recurrent neural networks is obtained when fusing the spectrogram and the bump scalogram, which is an improvement on the 61.0 % baseline result provided by the DCASE 2017 organisers. This result shows that extracted bump scalograms are capable of improving the classification accuracy,when fusing with a spectrogram-based system.
基金This paper is supported by the State Key Laboratory for Image Processing & Intelligent Control (No. TKLJ9903) National Defe
文摘Augmented reality is the merging of synthetic sensory information into a user's perception of a real environment. As one of the most important tasks in augmented scene modeling, terrain simplification research has gained more and more attention. In this paper, we mainly focus on point selection problem in terrain simplification using triangulated irregular network. Based on the analysis and comparison of traditional importance measures for each input point, we put forward a new importance measure based on local entropy. The results demonstrate that the local entropy criterion has a better performance than any traditional methods. In addition, it can effectively conquer the 'short-sight' problem associated with the traditional methods.
