MAUN:Memory-Augmented Deep Unfolding Network for Hyperspectral Image Reconstruction

Spectral compressive imaging has emerged as a powerful technique to collect the 3D spectral information as 2D measurements.The algorithm for restoring the original 3D hyperspectral images(HSIs)from compressive measurements is pivotal in the imaging process.Early approaches painstakingly designed networks to directly map compressive measurements to HSIs,resulting in the lack of interpretability without exploiting the imaging priors.While some recent works have introduced the deep unfolding framework for explainable reconstruction,the performance of these methods is still limited by the weak information transmission between iterative stages.In this paper,we propose a Memory-Augmented deep Unfolding Network,termed MAUN,for explainable and accurate HSI reconstruction.Specifically,MAUN implements a novel CNN scheme to facilitate a better extrapolation step of the fast iterative shrinkage-thresholding algorithm,introducing an extra momentum incorporation step for each iteration to alleviate the information loss.Moreover,to exploit the high correlation of intermediate images from neighboring iterations,we customize a cross-stage transformer(CSFormer)as the deep denoiser to simultaneously capture self-similarity from both in-stage and cross-stage features,which is the first attempt to model the long-distance dependencies between iteration stages.Extensive experiments demonstrate that the proposed MAUN is superior to other state-of-the-art methods both visually and metrically.Our code is publicly available at https://github.com/HuQ1an/MAUN.

Loop Closure Detection via Locality Preserving Matching With Global Consensus

A critical component of visual simultaneous localization and mapping is loop closure detection(LCD),an operation judging whether a robot has come to a pre-visited area.Concretely,given a query image(i.e.,the latest view observed by the robot),it proceeds by first exploring images with similar semantic information,followed by solving the relative relationship between candidate pairs in the 3D space.In this work,a novel appearance-based LCD system is proposed.Specifically,candidate frame selection is conducted via the combination of Superfeatures and aggregated selective match kernel(ASMK).We incorporate an incremental strategy into the vanilla ASMK to make it applied in the LCD task.It is demonstrated that this setting is memory-wise efficient and can achieve remarkable performance.To dig up consistent geometry between image pairs during loop closure verification,we propose a simple yet surprisingly effective feature matching algorithm,termed locality preserving matching with global consensus(LPM-GC).The major objective of LPM-GC is to retain the local neighborhood information of true feature correspondences between candidate pairs,where a global constraint is further designed to effectively remove false correspondences in challenging sceneries,e.g.,containing numerous repetitive structures.Meanwhile,we derive a closed-form solution that enables our approach to provide reliable correspondences within only a few milliseconds.The performance of the proposed approach has been experimentally evaluated on ten publicly available and challenging datasets.Results show that our method can achieve better performance over the state-of-the-art in both feature matching and LCD tasks.We have released our code of LPM-GC at https://github.com/jiayi-ma/LPM-GC.

Contrastive Learning for Blind Super-Resolution via A Distortion-Specific Network

Previous deep learning-based super-resolution(SR)methods rely on the assumption that the degradation process is predefined(e.g.,bicubic downsampling).Thus,their performance would suffer from deterioration if the real degradation is not consistent with the assumption.To deal with real-world scenarios,existing blind SR methods are committed to estimating both the degradation and the super-resolved image with an extra loss or iterative scheme.However,degradation estimation that requires more computation would result in limited SR performance due to the accumulated estimation errors.In this paper,we propose a contrastive regularization built upon contrastive learning to exploit both the information of blurry images and clear images as negative and positive samples,respectively.Contrastive regularization ensures that the restored image is pulled closer to the clear image and pushed far away from the blurry image in the representation space.Furthermore,instead of estimating the degradation,we extract global statistical prior information to capture the character of the distortion.Considering the coupling between the degradation and the low-resolution image,we embed the global prior into the distortion-specific SR network to make our method adaptive to the changes of distortions.We term our distortion-specific network with contrastive regularization as CRDNet.The extensive experiments on synthetic and realworld scenes demonstrate that our lightweight CRDNet surpasses state-of-the-art blind super-resolution approaches.

BaMBNet:A Blur-Aware Multi-Branch Network for Dual-Pixel Defocus Deblurring

Reducing the defocus blur that arises from the finite aperture size and short exposure time is an essential problem in computational photography.It is very challenging because the blur kernel is spatially varying and difficult to estimate by traditional methods.Due to its great breakthrough in low-level tasks,convolutional neural networks(CNNs)have been introdu-ced to the defocus deblurring problem and achieved significant progress.However,previous methods apply the same learned kernel for different regions of the defocus blurred images,thus it is difficult to handle nonuniform blurred images.To this end,this study designs a novel blur-aware multi-branch network(Ba-MBNet),in which different regions are treated differentially.In particular,we estimate the blur amounts of different regions by the internal geometric constraint of the dual-pixel(DP)data,which measures the defocus disparity between the left and right views.Based on the assumption that different image regions with different blur amounts have different deblurring difficulties,we leverage different networks with different capacities to treat different image regions.Moreover,we introduce a meta-learning defocus mask generation algorithm to assign each pixel to a proper branch.In this way,we can expect to maintain the information of the clear regions well while recovering the missing details of the blurred regions.Both quantitative and qualitative experiments demonstrate that our BaMBNet outperforms the state-of-the-art(SOTA)methods.For the dual-pixel defocus deblurring(DPD)-blur dataset,the proposed BaMBNet achieves 1.20 dB gain over the previous SOTA method in term of peak signal-to-noise ratio(PSNR)and reduces learnable parameters by 85%.The details of the code and dataset are available at https://github.com/junjun-jiang/BaMBNet.

Loop Closure Detection With Reweighting Net VLAD and Local Motion and Structure Consensus

Dear Editor,Loop closure detection(LCD)is an important module in simultaneous localization and mapping(SLAM).In this letter,we address the LCD task from the semantic aspect to the geometric one.To this end,a network termed as AttentionNetVLAD which can simultaneously extract global and local features is proposed.It leverages attentive selection for local features,coupling with reweighting the soft assignment in NetVLAD via the attention map for global features.Given a query image,candidate frames are first identified coarsely by retrieving similar global features in the database via hierarchical navigable small world(HNSW).As global features mainly summarize the semantic information of images and lead to compact representation,information about spatial arrangement of visual elements is lost.

Modelling and Backstepping Motion Control of the Aircraft Skin Inspection Robot

Aircraft skin health concerns whether the aircraft can fly safely.In this paper,an improved mechanical structure of the aircraft skin inspection robot was introduced.Considering that the aircraft skin surface is a curved environment,we assume that the curved environment is equivalent to an inclined plane with a change in inclination.Based on this assumption,the Cartesian dynamics model of the robot is established using the Lagrange method.In order to control the robot’s movement position accurately,a position backstepping control scheme for the aircraft skin inspection robot was presented.According to the dynamic model and taking into account the problems faced by the robot during its movement,a position constrained controller of the aircraft skin inspection robot is designed using the barrier Lyapunov function.Aiming at the disturbances in the robot,we adopt a fuzzy system to approximate the unknown dynamics related with system states.Finally,the simulation results of the designed position constrained controller were compared with the sliding mode controller,and prove the validity of the position constrained controller.

Self-Supervised Monocular Depth Estimation via Discrete Strategy and Uncertainty

Dear Editor,This letter is concerned with self-supervised monocular depth estimation.To estimate uncertainty simultaneously,we propose a simple yet effective strategy to learn the uncertainty for self-supervised monocular depth estimation with the discrete strategy that explicitly associates the prediction and the uncertainty to train the networks.Furthermore,we propose the uncertainty-guided feature fusion module to fully utilize the uncertainty information.Codes will be available at https://github.com/zhyever/Monocular-Depth-Estimation-Toolbox.Self-supervised monocular depth estimation methods turn into promising alternative trade-offs in both the training cost and the inference performance.However,compound losses that couple the depth and the pose lead to a dilemma of uncertainty calculation that is crucial for critical safety systems.To solve this issue,we propose a simple yet effective strategy to learn the uncertainty for self-supervised monocular depth estimation using the discrete bins that explicitly associate the prediction and the uncertainty to train the networks.This strategy is more pluggable without any additional changes to self-supervised training losses and improves model performance.Secondly,to further exert the uncertainty information,we propose the uncertainty-guided feature fusion module to refine the depth estimation.

Overhead-free Noise-tolerant Federated Learning: A New Baseline

Federated learning (FL) is a promising decentralized machine learning approach that enables multiple distributed clients to train a model jointly while keeping their data private. However, in real-world scenarios, the supervised training data stored in local clients inevitably suffer from imperfect annotations, resulting in subjective, inconsistent and biased labels. These noisy labels can harm the collaborative aggregation process of FL by inducing inconsistent decision boundaries. Unfortunately, few attempts have been made towards noise-tolerant federated learning, with most of them relying on the strategy of transmitting overhead messages to assist noisy labels detection and correction, which increases the communication burden as well as privacy risks. In this paper, we propose a simple yet effective method for noise-tolerant FL based on the well-established co-training framework. Our method leverages the inherent discrepancy in the learning ability of the local and global models in FL, which can be regarded as two complementary views. By iteratively exchanging samples with their high confident predictions, the two models “teach each other” to suppress the influence of noisy labels. The proposed scheme enjoys the benefit of overhead cost-free and can serve as a robust and efficient baseline for noise-tolerant federated learning. Experimental results demonstrate that our method outperforms existing approaches, highlighting the superiority of our method.

DepthFormer:Exploiting Long-range Correlation and Local Information for Accurate Monocular Depth Estimation

This paper aims to address the problem of supervised monocular depth estimation.We start with a meticulous pilot study to demonstrate that the long-range correlation is essential for accurate depth estimation.Moreover,the Transformer and convolution are good at long-range and close-range depth estimation,respectively.Therefore,we propose to adopt a parallel encoder architecture consisting of a Transformer branch and a convolution branch.The former can model global context with the effective attention mechanism and the latter aims to preserve the local information as the Transformer lacks the spatial inductive bias in modeling such contents.However,independent branches lead to a shortage of connections between features.To bridge this gap,we design a hierarchical aggregation and heterogeneous interaction module to enhance the Transformer features and model the affinity between the heterogeneous features in a set-to-set translation manner.Due to the unbearable memory cost introduced by the global attention on high-resolution feature maps,we adopt the deformable scheme to reduce the complexity.Extensive experiments on the KITTI,NYU,and SUN RGB-D datasets demonstrate that our proposed model,termed DepthFormer,surpasses state-of-the-art monocular depth estimation methods with prominent margins.The effectiveness of each proposed module is elaborately evaluated through meticulous and intensive ablation studies.

Image Enhancement via Associated Perturbation Removal and Texture Reconstruction Learning

Degradation under challenging conditions such as rain, haze, and low light not only diminishes content visibility, but also results in additional degradation side effects, including detail occlusion and color distortion. However, current technologies have barely explored the correlation between perturbation removal and background restoration, consequently struggling to generate high-naturalness content in challenging scenarios. In this paper, we rethink the image enhancement task from the perspective of joint optimization: Perturbation removal and texture reconstruction. To this end, we advise an efficient yet effective image enhancement model, termed the perturbation-guided texture reconstruction network(PerTeRNet). It contains two subnetworks designed for the perturbation elimination and texture reconstruction tasks, respectively. To facilitate texture recovery,we develop a novel perturbation-guided texture enhancement module(PerTEM) to connect these two tasks, where informative background features are extracted from the input with the guidance of predicted perturbation priors. To alleviate the learning burden and computational cost, we suggest performing perturbation removal in a sub-space and exploiting super-resolution to infer high-frequency background details. Our PerTeRNet has demonstrated significant superiority over typical methods in both quantitative and qualitative measures, as evidenced by extensive experimental results on popular image enhancement and joint detection tasks. The source code is available at https://github.com/kuijiang94/PerTeRNet.

基于深度学习的数字病理扫描系统单次曝光自动对焦方法

全切片数字成像(whole slide imaging, WSI)是病理切片数字化的核心技术,其自动对焦的速度和精度决定了WSI系统的性能.然而,传统的自动对焦方法需要拍摄多张离焦子图像创建图像堆栈,或者需要复杂的硬件调制光学系统,从而限制了WSI在实际场景中的应用.本文设计了基于深度学习的数字病理扫描系统单次曝光自动对焦方法,对子图像逐个进行网络虚拟自动对焦,将单张离焦子图像通过网络直接生成准焦子图像.本方法仅需要在任意离焦距离下的单次曝光拍照即可,从原理上避免了重复的对焦运动和相机曝光过程.实验结果表明,本方法具有高通量、高速度、低成本、实用性强、可线下处理等优点.

Comprehensive analysis of antibacterial and anti-hepatoma activity of metabolites from jujube fruit

Jujube is one of the most produced dried fruits in China,and it is also a traditional Chinese medicine that enhances immunity and has anti-cancer activity.Jujube fruit is rich in phenolic compounds,but few reports are available on its biological activities.To evaluate the biological activities of the phenolic compounds in dried jujube fruit,the composition and content of metabolites in dried jujube fruit were determined by metabolomics,and the antibacterial and anticancer activities of the phenolic compounds were analyzed in dried jujube fruit.The results showed that 463 compounds were identified in dried‘Junzao’fruit,including 102 phenolic compounds.The in vitro activity test showed that the jujube phenolic metabolites had extensive antibacterial effects and caused disruption and nuclear sclerosis of hepatocellular carcinoma(HepG2)cells.Canonical correlation analysis showed that total phenolic content,quercetin-3-rutinose,and procyanidin B1 were the main active antibacterial and anticancer components.The study provides data supporting the application of dried jujube fruit in the development of functional foods,pharmaceuticals and cosmetics.

MPIN:a macro-pixel integration network for light field super-resolution

Most existing light field(LF)super-resolution(SR)methods either fail to fully use angular information or have an unbalanced performance distribution because they use parts of views.To address these issues,we propose a novel integration network based on macro-pixel representation for the LF SR task,named MPIN.Restoring the entire LF image simultaneously,we couple the spatial and angular information by rearranging the four-dimensional LF image into a two-dimensional macro-pixel image.Then,two special convolutions are deployed to extract spatial and angular information,separately.To fully exploit spatial-angular correlations,the integration resblock is designed to merge the two kinds of information for mutual guidance,allowing our method to be angular-coherent.Under the macro-pixel representation,an angular shuffle layer is tailored to improve the spatial resolution of the macro-pixel image,which can effectively avoid aliasing.Extensive experiments on both synthetic and real-world LF datasets demonstrate that our method can achieve better performance than the state-of-the-art methods qualitatively and quantitatively.Moreover,the proposed method has an advantage in preserving the inherent epipolar structures of LF images with a balanced distribution of performance.

Systematic analysis of clinical relevance and molecular characterization of m^(6)A in COVID-19 patients

N6-methyladenosine(m6A)is the most prevalent post-transcriptional RNA modification in mRNA and long non-coding RNAs of eukaryotes,and its biological functions are mediated by m6A writers,erasers and readers.1 A nuclear methyltransferase complex consisting of METTL3,METTL14,WTAP,VIRMA,ZC3H13,RBM15(or RBM15B),YWHAG,TRA2A and CAPRIN1 catalyzes the m6A modifications,acting as m6A writers.1 m6A demethylase ALKBH5 as well as m6A demethylase FTO mediate the demethylation of m6As,acting as the m6A erasers.