Bridge the Gap Between Full-Reference and No-Reference:A Totally Full-Reference Induced Blind Image Quality Assessment via Deep Neural Networks

Blind image quality assessment(BIQA)is of fundamental importance in low-level computer vision community.Increasing interest has been drawn in exploiting deep neural networks for BIQA.Despite of the notable success achieved,there is a broad consensus that training deep convolutional neural networks(DCNN)heavily relies on massive annotated data.Unfortunately,BIQA is typically a small sample problem,resulting the generalization ability of BIQA severely restricted.In order to improve the accuracy and generalization ability of BIQA metrics,this work proposed a totally opinion-unaware BIQA in which no subjective annotations are involved in the training stage.Multiple full-reference image quality assessment(FR-IQA)metrics are employed to label the distorted image as a substitution of subjective quality annotation.A deep neural network(DNN)is trained to blindly predict the multiple FR-IQA score in absence of corresponding pristine image.In the end,a selfsupervised FR-IQA score aggregator implemented by adversarial auto-encoder pools the predictions of multiple FR-IQA scores into the final quality predicting score.Even though none of subjective scores are involved in the training stage,experimental results indicate that our proposed full reference induced BIQA framework is as competitive as state-of-the-art BIQA metrics.

Adaptive multi-sample-based photoacoustic tomography with imaging quality optimization

The energy of light exposed on human skin is compulsively limited for safety reasons which affects the power of photoacoustic （PA） signal and its signal-to-noise ratio （SNR） level. Thus, the final reconstructed PA image quality is degraded. This Letter proposes an adaptive multi-sample-based approach to enhance the SNR of PA signals and in addition, detailed information in rebuilt PA images that used to be buried in the noise can be distinguished. Both ex vivo and in vivo experiments are conducted to validate the effectiveness of our proposed method which provides its potential value in clinical trials.

Blind Image Quality Assessment by Pairwise Ranking Image Series

Image quality assessment(IQA)is constantly innovating,but there are still three types of stickers that have not been resolved:the“content sticker”-limitation of training set,the“annotation sticker”-subjective instability in opinion scores and the“distortion sticker”-disordered distortion settings.In this paper,a No-Reference Image Quality Assessment(NR IQA)approach is proposed to deal with the problems.For“content sticker”,we introduce the idea of pairwise comparison and generate a largescale ranking set to pre-train the network;For“annotation sticker”,the absolute noise-containing subjective scores are transformed into ranking comparison results,and we design an indirect unsupervised regression based on EigenValue Decomposition(EVD);For“distortion sticker”,we propose a perception-based distortion classification method,which makes the distortion types clear and refined.Experiments have proved that our NR IQA approach Experiments show that the algorithm performs well and has good generalization ability.Furthermore,the proposed perception based distortion classification method would be able to provide insights on how the visual related studies may be developed and to broaden our understanding of human visual system.

Lensless two-color ghost imaging from the perspective of coherent-mode representation

The coherent-mode representation theory is firstly used to analyze lensless two-color ghost imaging. A quite complicated expression about the point-spread function（PSF） needs to be given to analyze which wavelength has a stronger affect on imaging quality when the usual integral representation theory is used to ghost imaging. Unlike this theory, the coherent-mode representation theory shows that imaging quality depends crucially on the distribution of the decomposition coefficients of the object imaged in a two-color ghost imaging. The analytical expression of the decomposition coefficients of the object is unconcerned with the wavelength of the light used in the reference arm, but has relevance with the wavelength in the object arm. In other words, imaging quality of two-color ghost imaging depends primarily on the wavelength of the light illuminating the object. Our simulation results also demonstrate this conclusion.

Target acquisition performance in the presence of JPEG image compression

This paper presents an investigation on the effect of JPEG compression on the similarity between the target image and the background,where the similarity is further used to determine the degree of clutter in the image.Four new clutter metrics based on image quality assessment are introduced,among which the Haar wavelet-based perceptual similarity index,known as HaarPSI,provides the best target acquisition prediction results.It is shown that the similarity between the target and the background at the boundary between visually lossless and visually lossy compression does not change significantly compared to the case when an uncompressed image is used.In future work,through subjective tests,it is necessary to check whether this presence of compression at the threshold of just noticeable differences will affect the human target acquisition performance.Similarity values are compared with the results of subjective tests of the well-known target Search_2 database,where the degree of agreement between objective and subjective scores,measured through linear correlation,reached a value of 90%.

Compressive imaging based on multi-scale modulation and reconstruction in spatial frequency domain

Imaging quality is a critical component of compressive imaging in real applications. In this study, we propose a compressive imaging method based on multi-scale modulation and reconstruction in the spatial frequency domain. Theoretical analysis and simulation show the relation between the measurement matrix resolution and compressive sensing(CS)imaging quality. The matrix design is improved to provide multi-scale modulations, followed by individual reconstruction of images of different spatial frequencies. Compared with traditional single-scale CS imaging, the multi-scale method provides high quality imaging in both high and low frequencies, and effectively decreases the overall reconstruction error.Experimental results confirm the feasibility of this technique, especially at low sampling rate. The method may thus be helpful in promoting the implementation of compressive imaging in real applications.

Study on analytical noise propagation in convolutional neural network methods used in computed tomography imaging

Neural network methods have recently emerged as a hot topic in computed tomography(CT) imaging owing to their powerful fitting ability;however, their potential applications still need to be carefully studied because their results are often difficult to interpret and are ambiguous in generalizability. Thus, quality assessments of the results obtained from a neural network are necessary to evaluate the neural network. Assessing the image quality of neural networks using traditional objective measurements is not appropriate because neural networks are nonstationary and nonlinear. In contrast, subjective assessments are trustworthy, although they are time-and energy-consuming for radiologists. Model observers that mimic subjective assessment require the mean and covariance of images, which are calculated from numerous image samples;however, this has not yet been applied to the evaluation of neural networks. In this study, we propose an analytical method for noise propagation from a single projection to efficiently evaluate convolutional neural networks(CNNs) in the CT imaging field. We propagate noise through nonlinear layers in a CNN using the Taylor expansion. Nesting of the linear and nonlinear layer noise propagation constitutes the covariance estimation of the CNN. A commonly used U-net structure is adopted for validation. The results reveal that the covariance estimation obtained from the proposed analytical method agrees well with that obtained from the image samples for different phantoms, noise levels, and activation functions, demonstrating that propagating noise from only a single projection is feasible for CNN methods in CT reconstruction. In addition, we use covariance estimation to provide three measurements for the qualitative and quantitative performance evaluation of U-net. The results indicate that the network cannot be applied to projections with high noise levels and possesses limitations in terms of efficiency for processing low-noise projections. U-net is more effective in improving the image quality of smooth regions compared with that of the edge. LeakyReLU outperforms Swish in terms of noise reduction.

Multi-Modality Medical Image Fusion Based on Wavelet Analysis and Quality Evaluation

Multi-modality medical image fusion has more and more important applications in medical image analysis and understanding. In this paper, we develop and apply a multi-resolution method based on wavelet pyramid to fuse medical images from different modalities such as PET-MRI and CT-MRI. In particular, we evaluate the different fusion results when applying different selection rules and obtain optimum combination of fusion parameters.

Image Tampering Detection Using No-Reference Image Quality Metrics

In this paper,a new approach is proposed to determine whether the content of an image is authentic or modified with a focus on detecting complex image tampering.Detecting image tampering without any prior information of the original image is a challenging problem since unknown diverse manipulations may have different characteristics and so do various formats of images.Our principle is that image processing,no matter how complex,may affect image quality,so image quality metrics can be used to distinguish tampered images.In particular,based on the alteration of image quality in modified blocks,the proposed method can locate the tampered areas.Referring to four types of effective no-reference image quality metrics,we obtain 13 features to present an image.The experimental results show that the proposed method is very promising on detecting image tampering and locating the locally tampered areas especially in realistic scenarios.

NEW VISUAL PERCEPTUAL POOLING STRATEGY FOR IMAGE QUALITY ASSESSMENT

Most of Image Quality Assessment (IQA) metrics consist of two processes. In the first process, quality map of image is measured locally. In the second process, the last quality score is converted from the quality map by using the pooling strategy. The first process had been made effective and significant progresses, while the second process was always done in simple ways. In the second process of the pooling strategy, the optimal perceptual pooling weights should be determined and computed according to Human Visual System (HVS). Thus, a reliable spatial pooling mathematical model based on HVS is an important issue worthy of study. In this paper, a new Visual Perceptual Pooling Strategy (VPPS) for IQA is presented based on contrast sensitivity and luminance sensitivity of HVS. Experimental results with the LIVE database show that the visual perceptual weights, obtained by the proposed pooling strategy, can effectively and significantly improve the performances of the IQA metrics with Mean Structural SIMilarity (MSSIM) or Phase Quantization Code (PQC). It is confirmed that the proposed VPPS demonstrates promising results for improving the performances of existing IQA metrics.

A METHOD OF IMAGE QUALITY ASSESSMENT FOR COMPRESSIVE SAMPLING VIDEO TRANSMISSION

Based on compressive sampling transmission model, we demonstrate here a method of quality evaluation for the reconstruction images, which is promising for the transmission of unstructured signal with reduced dimension. By this method, the auxiliary information of the recovery image quality is obtained as a feedback to control number of measurements from compressive sampling video stream. Therefore, the number of measurements can be easily derived at the condition of the absence of information sparsity, and the recovery image quality is effectively improved. Theoretical and experimental results show that this algorithm can estimate the quality of images effectively and is in well consistency with the traditional objective evaluation algorithm.

Multi-Scale Blind Image Quality Predictor Based on Pyramidal Convolution

Traditional image quality assessment methods use the hand-crafted features to predict the image quality score,which cannot perform well in many scenes.Since deep learning promotes the development of many computer vision tasks,many IQA methods start to utilize the deep convolutional neural networks(CNN)for IQA task.In this paper,a CNN-based multi-scale blind image quality predictor is proposed to extract more effectivity multi-scale distortion features through the pyramidal convolution,which consists of two tasks:A distortion recognition task and a quality regression task.For the first task,image distortion type is obtained by the fully connected layer.For the second task,the image quality score is predicted during the distortion recognition progress.Experimental results on three famous IQA datasets show that the proposed method has better performance than the previous traditional algorithms for quality prediction and distortion recognition.

Toward a Neurophysiological Measure of Image Quality Perception Based on Algebraic Topology Analysis

The bandwidth of internet connections is still a bottleneck when transmitting large amounts of images,making the image quality assessment essential.Neurophysiological assessment of image quality has highlight advantages for it does not interfere with natural viewing behavior.However,in JPEG compression,the previous study is hard to tell the difference between the electroencephalogram(EEG)evoked by different quality images.In this paper,we propose an EEG analysis approach based on algebraic topology analysis,and the result shows that the difference between Euler characteristics of EEG evoked by different distortion images is striking both in the alpha and beta band.Moreover,we further discuss the relationship between the images and the EEG signals,and the results implied that the algebraic topological properties of images are consistent with that of brain perception,which is possible to give birth to braininspired image compression based on algebraic topological features.In general,an algebraic topologybased approach was proposed in this paper to analyze the perceptual characteristics of image quality,which will be beneficial to provide a reliable score for data compression in the network and improve the network transmission capacity.

Geometric Calibration and Image Quality Assessment of High Resolution Dual-Camera Satellite

The evaluation of geometric calibration accuracy of high resolution satellite images has been increasingly recognized in recent years.In order to evaluate geometric accuracy for dual-camera satellite images based on the ground control points(GCP),a rigorous geometric imaging model,which was based on the collinear equation of the probe directional angle and the optimized tri-axial attitude determination(TRIAD)algorithm,is presented.Two reliable test fields in Tianjin and Jinan(China)were utilized for geometric accuracy validation of Pakistan Remote Sensing Satellite-1.The experimental results demonstrate a certain deviation of the on-orbit calibration result from the initial design values of the calibration parameters.Therefore,on-orbit geometric calibration is necessary for optical satellite imagery.Within this research,the geometrical performances including positioning accuracy without/with GCP and band registration of the dual-camera satellite were analyzed in detail,and the results of geometric image quality are assessed and discussed.As a result,it is feasible and necessary to establish such a geometric calibration model to evaluate the geometric quality of dual-camera satellite.

Blind Image Quality Assessment Based on Hybrid Fuzzy-Genetic Technique

A new method for no-reference image quality assessment based on hybrid fuzzy-genetic technique is proposed. Noise variance and edge sharpness level of the restored image are two basic metrics for assessing the performance of the restoration algorithm, then a fuzzy if-then inference system is developed to combine the two metrics to get a final quality score, and the parameters of the fuzzy membership function are trained with genetic algorithms. Experiments results show that the image quality score correlates well with mean opinion score and the proposed approach is robust and effective.

Perceptual quality assessment of panoramic stitched contents for immersive applications:a prospective survey

The recent advancements in the field of Virtual Reality(VR)and Augmented Reality(AR)have a substantial impact on modern day technology by digitizing each and everything related to human life and open the doors to the next generation Software Technology(Soft Tech).VR and AR technology provide astonishing immersive contents with the help of high quality stitched panoramic contents and 360°imagery that widely used in the education,gaming,entertainment,and production sector.The immersive quality of VR and AR contents are greatly dependent on the perceptual quality of panoramic or 360°images,in fact a minor visual distortion can significantly degrade the overall quality.Thus,to ensure the quality of constructed panoramic contents for VR and AR applications,numerous Stitched Image Quality Assessment(SIQA)methods have been proposed to assess the quality of panoramic contents before using in VR and AR.In this survey,we provide a detailed overview of the SIQA literature and exclusively focus on objective SIQA methods presented till date.For better understanding,the objective SIQA methods are classified into two classes namely Full-Reference SIQA and No-Reference SIQA approaches.Each class is further categorized into traditional and deep learning-based methods and examined their performance for SIQA task.Further,we shortlist the publicly available benchmark SIQA datasets and evaluation metrices used for quality assessment of panoramic contents.In last,we highlight the current challenges in this area based on the existing SIQA methods and suggest future research directions that need to be target for further improvement in SIQA domain.

Computed Tomography Protocol Optimisation for Pediatric Head Trauma: Radiation Dose and Image Quality Assessment

Purpose: Children are sometimes examined with Computed Tomography protocols designed for adults, leading to radiation doses higher than necessary. Lack of optimisation could lead to image quality higher than what is needed for diagnostic purposes with associated high doses to patients. Optimising the protocols for paediatric head trauma CT imaging will reduce radiation dose. Objective: The study aimed to optimise radiation dose and assess the image quality for a set of protocols by evaluating noise, a contrast to noise ratio, modulation transfer function and noise power spectrum. Methods: Somaton Sensation 64 was used to scan the head of an anthropomorphic phantom with a set of protocols. ImageJ software was used to analyse the paediatric head image from the scanner. IMPACTSCAN dosimeter software was used to evaluate the radiation dose to the various organs in the head. MATLAB was used to analyse the Modulation Transfer Function and the Noise Power. Results: The estimated Computed Tomography Dose Index volume (CTDIvol) increased with increasing tube current and tube voltage. The high pitch of 0.9 gave a lower dose than the 0.5 pitch. The eye lens received the highest radiation dose (39.2 mGy) whiles the thyroid received the least radiation dose (13.7 mGy). There was an increase in noise (62.46) when the H60 kernel was used and a lower noise (8.829) was noticed when the H30 kernel was used. Conclusion: The results obtained show that the H30 kernel (smooth kernel) gave higher values for noise and contrast to noise ratio (CNR) than the H60 kernel (sharp kernel). The H60 kernel produced high values for the modulation transfer function (MTF) and noise power spectrum (NPS). The eye lens received the highest radiation dose.

No-Reference Stereo Image Quality Assessment Based on Transfer Learning

In order to apply the deep learning to the stereo image quality evaluation,two problems need to be solved:The first one is that we have a bit of training samples,another is how to input the dimensional image’s left view or right view.In this paper,we transfer the 2D image quality evaluation model to the stereo image quality evaluation,and this method solves the first problem;use the method of principal component analysis is used to fuse the left and right views into an input image in order to solve the second problem.At the same time,the input image is preprocessed by phase congruency transformation,which further improves the performance of the algorithm.The structure of the deep convolution neural network consists of four convolution layers and three maximum pooling layers and two fully connected layers.The experimental results on LIVE3D image database show that the prediction quality score of the model is in good agreement with the subjective evaluation value.

A two-dimensional MoS_(2) array based on artificial neural network learning for high-quality imaging

As the basis of machine vision,the biomimetic image sensing devices are the eyes of artificial intelligence.In recent years,with the development of two-dimensional(2D)materials,many new optoelectronic devices are developed for their outstanding performance.However,there are still little sensing arrays based on 2D materials with high imaging quality,due to the poor uniformity of pixels caused by material defects and fabrication technique.Here,we propose a 2D MoS_(2)sensing array based on artificial neural network(ANN)learning.By equipping the MoS_(2)sensing array with a“brain”(ANN),the imaging quality can be effectively improved.In the test,the relative standard deviation(RSD)between pixels decreased from about 34.3%to 6.2%and 5.49%after adjustment by the back propagation(BP)and Elman neural networks,respectively.The peak signal to noise ratio(PSNR)and structural similarity(SSIM)of the image are improved by about 2.5 times,which realizes the re-recognition of the distorted image.This provides a feasible approach for the application of 2D sensing array by integrating ANN to achieve high quality imaging.