More Than Lightening:A Self-Supervised Low-Light Image Enhancement Method Capable for Multiple Degradations

Low-light images suffer from low quality due to poor lighting conditions,noise pollution,and improper settings of cameras.To enhance low-light images,most existing methods rely on normal-light images for guidance but the collection of suitable normal-light images is difficult.In contrast,a self-supervised method breaks free from the reliance on normal-light data,resulting in more convenience and better generalization.Existing self-supervised methods primarily focus on illumination adjustment and design pixel-based adjustment methods,resulting in remnants of other degradations,uneven brightness and artifacts.In response,this paper proposes a self-supervised enhancement method,termed as SLIE.It can handle multiple degradations including illumination attenuation,noise pollution,and color shift,all in a self-supervised manner.Illumination attenuation is estimated based on physical principles and local neighborhood information.The removal and correction of noise and color shift removal are solely realized with noisy images and images with color shifts.Finally,the comprehensive and fully self-supervised approach can achieve better adaptability and generalization.It is applicable to various low light conditions,and can reproduce the original color of scenes in natural light.Extensive experiments conducted on four public datasets demonstrate the superiority of SLIE to thirteen state-of-the-art methods.Our code is available at https://github.com/hanna-xu/SLIE.

Edge-Federated Self-Supervised Communication Optimization Framework Based on Sparsification and Quantization Compression

The federated self-supervised framework is a distributed machine learning method that combines federated learning and self-supervised learning, which can effectively solve the problem of traditional federated learning being difficult to process large-scale unlabeled data. The existing federated self-supervision framework has problems with low communication efficiency and high communication delay between clients and central servers. Therefore, we added edge servers to the federated self-supervision framework to reduce the pressure on the central server caused by frequent communication between both ends. A communication compression scheme using gradient quantization and sparsification was proposed to optimize the communication of the entire framework, and the algorithm of the sparse communication compression module was improved. Experiments have proved that the learning rate changes of the improved sparse communication compression module are smoother and more stable. Our communication compression scheme effectively reduced the overall communication overhead.

Boosting battery state of health estimation based on self-supervised learning

State of health(SoH) estimation plays a key role in smart battery health prognostic and management.However,poor generalization,lack of labeled data,and unused measurements during aging are still major challenges to accurate SoH estimation.Toward this end,this paper proposes a self-supervised learning framework to boost the performance of battery SoH estimation.Different from traditional data-driven methods which rely on a considerable training dataset obtained from numerous battery cells,the proposed method achieves accurate and robust estimations using limited labeled data.A filter-based data preprocessing technique,which enables the extraction of partial capacity-voltage curves under dynamic charging profiles,is applied at first.Unsupervised learning is then used to learn the aging characteristics from the unlabeled data through an auto-encoder-decoder.The learned network parameters are transferred to the downstream SoH estimation task and are fine-tuned with very few sparsely labeled data,which boosts the performance of the estimation framework.The proposed method has been validated under different battery chemistries,formats,operating conditions,and ambient.The estimation accuracy can be guaranteed by using only three labeled data from the initial 20% life cycles,with overall errors less than 1.14% and error distribution of all testing scenarios maintaining less than 4%,and robustness increases with aging.Comparisons with other pure supervised machine learning methods demonstrate the superiority of the proposed method.This simple and data-efficient estimation framework is promising in real-world applications under a variety of scenarios.

Research on Self-Supervised Comparative Learning for Computer Vision

In recent years,self-supervised learning which does not require a large number of manual labels generate supervised signals through the data itself to attain the characterization learning of samples.Self-supervised learning solves the problem of learning semantic features from unlabeled data,and realizes pre-training of models in large data sets.Its significant advantages have been extensively studied by scholars in recent years.There are usually three types of self-supervised learning:"Generative,Contrastive,and GeneTative-Contrastive."The model of the comparative learning method is relatively simple,and the performance of the current downstream task is comparable to that of the supervised learning method.Therefore,we propose a conceptual analysis framework:data augmentation pipeline,architectures,pretext tasks,comparison methods,semisupervised fine-tuning.Based on this conceptual framework,we qualitatively analyze the existing comparative self-supervised learning methods for computer vision,and then further analyze its performance at different stages,and finally summarize the research status of sei supervised comparative learning methods in other fields.

The potential of self-supervised networks for random noise suppression in seismic data

Noise suppression is an essential step in many seismic processing workflows.A portion of this noise,particularly in land datasets,presents itself as random noise.In recent years,neural networks have been successfully used to denoise seismic data in a supervised fashion.However,supervised learning always comes with the often unachievable requirement of having noisy-clean data pairs for training.Using blind-spot networks,we redefine the denoising task as a self-supervised procedure where the network uses the surrounding noisy samples to estimate the noise-free value of a central sample.Based on the assumption that noise is statistically independent between samples,the network struggles to predict the noise component of the sample due to its randomicity,whilst the signal component is accurately predicted due to its spatio-temporal coherency.Illustrated on synthetic examples,the blind-spot network is shown to be an efficient denoiser of seismic data contaminated by random noise with minimal damage to the signal;therefore,providing improvements in both the image domain and down-the-line tasks,such as post-stack inversion.To conclude our study,the suggested approach is applied to field data and the results are compared with two commonly used random denoising techniques:FX-deconvolution and sparsity-promoting inversion by Curvelet transform.By demonstrating that blind-spot networks are an efficient suppressor of random noise,we believe this is just the beginning of utilising self-supervised learning in seismic applications.

The enlightenment of artificial intelligence large-scale model on the research of intelligent eye diagnosis in traditional Chinese medicine

Eye diagnosis is a method for inspecting systemic diseases and syndromes by observing the eyes.With the development of intelligent diagnosis in traditional Chinese medicine(TCM);artificial intelligence(AI)can improve the accuracy and efficiency of eye diagnosis.However;the research on intelligent eye diagnosis still faces many challenges;including the lack of standardized and precisely labeled data;multi-modal information analysis;and artificial in-telligence models for syndrome differentiation.The widespread application of AI models in medicine provides new insights and opportunities for the research of eye diagnosis intelli-gence.This study elaborates on the three key technologies of AI models in the intelligent ap-plication of TCM eye diagnosis;and explores the implications for the research of eye diagno-sis intelligence.First;a database concerning eye diagnosis was established based on self-su-pervised learning so as to solve the issues related to the lack of standardized and precisely la-beled data.Next;the cross-modal understanding and generation of deep neural network models to address the problem of lacking multi-modal information analysis.Last;the build-ing of data-driven models for eye diagnosis to tackle the issue of the absence of syndrome dif-ferentiation models.In summary;research on intelligent eye diagnosis has great potential to be applied the surge of AI model applications.

The 3-billion fossil question:How to automate classification of microfossils

Microfossil classification is an important discipline in subsurface exploration,for both oil&gas and Carbon Capture and Storage(CCS).The abundance and distribution of species found in sedimentary rocks provide valuable information about the age and depositional environment.However,the analysis is difficult and consuming,time-as it is based on manual work by human experts.Attempts to automate this process face two key challenges:(1)the input data are very large-our dataset is projected to grow to 3 billion microfossils,and(2)there are not enough labeled data to use the standard procedure of training a deep learning classifier.We propose an efficient pipeline for processing and grouping fossils by genus,or even species,from microscope slides using self-supervised learning.First we show how to efficiently extract crops from whole slide images by adapting previously trained object detection algorithms.Second,we provide a comparison of a range of self-supervised learning methods to classify and identify microfossils from very few labels.We obtain excellent results with both convolutional neural networks and vision transformers fine-tuned by self-supervision.Our approach is fast and computationally light,providing a handy tool for geologists working with microfossils.

Unlocking the potential of unlabeled data:Self-supervised machine learning for battery aging diagnosis with real-world field data

Accurate aging diagnosis is crucial for the health and safety management of lithium-ion batteries in electric vehicles.Despite significant advancements achieved by data-driven methods,diagnosis accuracy remains constrained by the high costs of check-up tests and the scarcity of labeled data.This paper presents a framework utilizing self-supervised machine learning to harness the potential of unlabeled data for diagnosing battery aging in electric vehicles during field operations.We validate our method using battery degradation datasets collected over more than two years from twenty real-world electric vehicles.Our analysis comprehensively addresses cell inconsistencies,physical interpretations,and charging uncertainties in real-world applications.This is achieved through self-supervised feature extraction using random short charging sequences in the main peak of incremental capacity curves.By leveraging inexpensive unlabeled data in a self-supervised approach,our method demonstrates improvements in average root mean square errors of 74.54%and 60.50%in the best and worst cases,respectively,compared to the supervised benchmark.This work underscores the potential of employing low-cost unlabeled data with self-supervised machine learning for effective battery health and safety management in realworld scenarios.

A Novel Self-Supervised Learning Network for Binocular Disparity Estimation

Two-dimensional endoscopic images are susceptible to interferences such as specular reflections and monotonous texture illumination,hindering accurate three-dimensional lesion reconstruction by surgical robots.This study proposes a novel end-to-end disparity estimation model to address these challenges.Our approach combines a Pseudo-Siamese neural network architecture with pyramid dilated convolutions,integrating multi-scale image information to enhance robustness against lighting interferences.This study introduces a Pseudo-Siamese structure-based disparity regression model that simplifies left-right image comparison,improving accuracy and efficiency.The model was evaluated using a dataset of stereo endoscopic videos captured by the Da Vinci surgical robot,comprising simulated silicone heart sequences and real heart video data.Experimental results demonstrate significant improvement in the network’s resistance to lighting interference without substantially increasing parameters.Moreover,the model exhibited faster convergence during training,contributing to overall performance enhancement.This study advances endoscopic image processing accuracy and has potential implications for surgical robot applications in complex environments.

iPAS:A deep Monte Carlo Tree Search-based intelligent pilot-power allocation scheme for massive MIMO system

Massive Multiple-Input-Multiple-Output(MIMO)is a promising technology to meet the demand for the connection of massive devices and high data capacity for mobile networks in the next generation communication system.However,due to the massive connectivity of mobile devices,the pilot contamination problem will severely degrade the communication quality and spectrum efficiency of the massive MIMO system.We propose a deep Monte Carlo Tree Search(MCTS)-based intelligent Pilot-power Allocation Scheme(iPAS)to address this issue.The core of iPAS is a multi-task deep reinforcement learning algorithm that can automatically learn the radio environment and make decisions on the pilot sequence and power allocation to maximize the spectrum efficiency with self-play training.To accelerate the searching convergence,we introduce a Deep Neural Network(DNN)to predict the pilot sequence and power allocation actions.The DNN is trained in a self-supervised learning manner,where the training data is generated from the searching process of the MCTS algorithm.Numerical results show that our proposed iPAS achieves a better Cumulative Distribution Function(CDF)of the ergodic spectral efficiency compared with the previous suboptimal algorithms.

A Multi-Level Circulant Cross-Modal Transformer for Multimodal Speech Emotion Recognition

Speech emotion recognition,as an important component of humancomputer interaction technology,has received increasing attention.Recent studies have treated emotion recognition of speech signals as a multimodal task,due to its inclusion of the semantic features of two different modalities,i.e.,audio and text.However,existing methods often fail in effectively represent features and capture correlations.This paper presents a multi-level circulant cross-modal Transformer(MLCCT)formultimodal speech emotion recognition.The proposed model can be divided into three steps,feature extraction,interaction and fusion.Self-supervised embedding models are introduced for feature extraction,which give a more powerful representation of the original data than those using spectrograms or audio features such as Mel-frequency cepstral coefficients(MFCCs)and low-level descriptors(LLDs).In particular,MLCCT contains two types of feature interaction processes,where a bidirectional Long Short-term Memory(Bi-LSTM)with circulant interaction mechanism is proposed for low-level features,while a two-stream residual cross-modal Transformer block is appliedwhen high-level features are involved.Finally,we choose self-attention blocks for fusion and a fully connected layer to make predictions.To evaluate the performance of our proposed model,comprehensive experiments are conducted on three widely used benchmark datasets including IEMOCAP,MELD and CMU-MOSEI.The competitive results verify the effectiveness of our approach.

SSL Depth: self-supervised learning enables 16× speedup in confocal microscopy-based 3D surface imaging [Invited]

In scientific and industrial research, three-dimensional (3D) imaging, or depth measurement, is a critical tool that provides detailed insight into surface properties. Confocal microscopy, known for its precision in surface measurements, plays a key role in this field. However, 3D imaging based on confocal microscopy is often challenged by significant data requirements and slow measurement speeds. In this paper, we present a novel self-supervised learning algorithm called SSL Depth that overcomes these challenges. Specifically, our method exploits the feature learning capabilities of neural networks while avoiding the need for labeled data sets typically associated with supervised learning approaches. Through practical demonstrations on a commercially available confocal microscope, we find that our method not only maintains higher quality, but also significantly reduces the frequency of the z-axis sampling required for 3D imaging. This reduction results in a remarkable 16×measurement speed, with the potential for further acceleration in the future. Our methodological advance enables highly efficient and accurate 3D surface reconstructions, thereby expanding the potential applications of confocal microscopy in various scientific and industrial fields.

What is Discussed about COVID-19:A Multi-Modal Framework for Analyzing Microblogs from Sina Weibo without Human Labeling

Starting from late 2019,the new coronavirus disease(COVID-19)has become a global crisis.With the development of online social media,people prefer to express their opinions and discuss the latest news online.We have witnessed the positive influence of online social media,which helped citizens and governments track the development of this pandemic in time.It is necessary to apply artificial intelligence(AI)techniques to online social media and automatically discover and track public opinions posted online.In this paper,we take Sina Weibo,the most widely used online social media in China,for analysis and experiments.We collect multi-modal microblogs about COVID-19 from 2020/1/1 to 2020/3/31 with a web crawler,including texts and images posted by users.In order to effectively discover what is being discussed about COVID-19 without human labeling,we propose a unified multi-modal framework,including an unsupervised short-text topic model to discover and track bursty topics,and a self-supervised model to learn image features so that we can retrieve related images about COVID-19.Experimental results have shown the effectiveness and superiority of the proposed models,and also have shown the considerable application prospects for analyzing and tracking public opinions about COVID-19.

On Robust Cross-view Consistency in Self-supervised Monocular Depth Estimation

Remarkable progress has been made in self-supervised monocular depth estimation (SS-MDE) by exploring cross-view consistency, e.g., photometric consistency and 3D point cloud consistency. However, they are very vulnerable to illumination variance, occlusions, texture-less regions, as well as moving objects, making them not robust enough to deal with various scenes. To address this challenge, we study two kinds of robust cross-view consistency in this paper. Firstly, the spatial offset field between adjacent frames is obtained by reconstructing the reference frame from its neighbors via deformable alignment, which is used to align the temporal depth features via a depth feature alignment (DFA) loss. Secondly, the 3D point clouds of each reference frame and its nearby frames are calculated and transformed into voxel space, where the point density in each voxel is calculated and aligned via a voxel density alignment (VDA) loss. In this way, we exploit the temporal coherence in both depth feature space and 3D voxel space for SS-MDE, shifting the “point-to-point” alignment paradigm to the “region-to-region” one. Compared with the photometric consistency loss as well as the rigid point cloud alignment loss, the proposed DFA and VDA losses are more robust owing to the strong representation power of deep features as well as the high tolerance of voxel density to the aforementioned challenges. Experimental results on several outdoor benchmarks show that our method outperforms current state-of-the-art techniques. Extensive ablation study and analysis validate the effectiveness of the proposed losses, especially in challenging scenes. The code and models are available at https://github.com/sunnyHelen/RCVC-depth.

Audio Mixing Inversion via Embodied Self-supervised Learning

Audio mixing is a crucial part of music production.For analyzing or recreating audio mixing,it is of great importance to conduct research on estimating mixing parameters used to create mixdowns from music recordings,i.e.,audio mixing inversion.However,approaches of audio mixing inversion are rarely explored.A method of estimating mixing parameters from raw tracks and a stereo mixdown via embodied self-supervised learning is presented.In this work,several commonly used audio effects including gain,pan,equalization,reverb,and compression,are taken into consideration.This method is able to learn an inference neural network that takes a stereo mixdown and the raw audio sources as input and estimate mixing parameters used to create the mixdown by iteratively sampling and training.During the sampling step,the inference network predicts a set of mixing parameters,which is sampled and fed to an audio-processing framework to generate audio data for the training step.During the training step,the same network used in the sampling step is optimized with the sampled data generated from the sampling step.This method is able to explicitly model the mixing process in an interpretable way instead of using a black-box neural network model.A set of objective measures are used for evaluation.The experimental results show that this method has better performance than current state-of-the-art methods.

A data-efficient self-supervised deep learning model for design and characterization of nanophotonic structures

With its tremendous success in many machine learning and pattern recognition tasks,deep learning,as one type of data-driven models,has also led to many breakthroughs in other disciplines including physics,chemistry and material science.Nevertheless,the supremacy of deep learning over conventional optimization approaches heavily depends on the huge amount of data collected in advance to train the model,which is a common bottleneck of such a data-driven technique.In this work,we present a comprehensive deep learning model for the design and characterization of nanophotonic structures,where a self-supervised learning mechanism is introduced to alleviate the burden of data acquisition.Taking reflective metasurfaces as an example,we demonstrate that the self-supervised deep learning model can effectively utilize randomly generated unlabeled data during training,with the total test loss and prediction accuracy improved by about 15%compared with the fully supervised counterpart.The proposed self-supervised learning scheme provides an efficient solution for deep learning models in some physics-related tasks where labeled data are limited or expensive to collect.

Self-supervised learning-based oil spill detection of hyperspectral images

Oil spill monitoring in remote sensing field has become a very popular technology to detect the spatial distribution of polluted regions.However,previous studies mainly focus on the supervised detection technologies,which requires a large number of high-quality training set.To solve this problem,we propose a self-supervised learning method to learn the deep neural network from unlabelled hyperspectral data for oil spill detection,which consists of three parts:data augmentation,unsupervised deep feature learning,and oil spill detection network.First,the original image is augmented with spectral and spatial transformation to improve robustness of the self-supervised model.Then,the deep neural networks are trained on the augmented data without label information to produce the high-level semantic features.Finally,the pre-trained parameters are transferred to establish a neural network classifier to obtain the detection result,where a contrastive loss is developed to fine-tune the learned parameters so as to improve the generalization ability of the proposed method.Experiments performed on ten oil spill datasets reveal that the proposed method obtains promising detection performance with respect to other state-of-the-art hyperspectral detection approaches.

A Review of Predictive and Contrastive Self-supervised Learning for Medical Images

Over the last decade, supervised deep learning on manually annotated big data has been progressing significantly on computer vision tasks. But, the application of deep learning in medical image analysis is limited by the scarcity of high-quality annotated medical imaging data. An emerging solution is self-supervised learning (SSL), among which contrastive SSL is the most successful approach to rivalling or outperforming supervised learning. This review investigates several state-of-the-art contrastive SSL algorithms originally on natural images as well as their adaptations for medical images, and concludes by discussing recent advances, current limitations, and future directions in applying contrastive SSL in the medical domain.

Contrastive Self-supervised Representation Learning Using Synthetic Data

Learning discriminative representations with deep neural networks often relies on massive labeled data, which is expensive and difficult to obtain in many real scenarios. As an alternative, self-supervised learning that leverages input itself as supervision is strongly preferred for its soaring performance on visual representation learning. This paper introduces a contrastive self-supervised framework for learning generalizable representations on the synthetic data that can be obtained easily with complete controllability.Specifically, we propose to optimize a contrastive learning task and a physical property prediction task simultaneously. Given the synthetic scene, the first task aims to maximize agreement between a pair of synthetic images generated by our proposed view sampling module, while the second task aims to predict three physical property maps, i.e., depth, instance contour maps, and surface normal maps. In addition, a feature-level domain adaptation technique with adversarial training is applied to reduce the domain difference between the realistic and the synthetic data. Experiments demonstrate that our proposed method achieves state-of-the-art performance on several visual recognition datasets.

Self-Supervised Task Augmentation for Few-Shot Intent Detection

Few-shot intent detection is a practical challenge task,because new intents are frequently emerging and collecting large-scale data for them could be costly.Meta-learning,a promising technique for leveraging data from previous tasks to enable efficient learning of new tasks,has been a popular way to tackle this problem.However,the existing meta-learning models have been evidenced to be overfitting when the meta-training tasks are insufficient.To overcome this challenge,we present a novel self-supervised task augmentation with meta-learning framework,namely STAM.Firstly,we introduce the task augmentation,which explores two different strategies and combines them to extend meta-training tasks.Secondly,we devise two auxiliary losses for integrating self-supervised learning into meta-learning to learn more generalizable and transferable features.Experimental results show that STAM can achieve consistent and considerable performance improvement to existing state-of-the-art methods on four datasets.