M3SC:A Generic Dataset for Mixed Multi-Modal(MMM)Sensing and Communication Integration

The sixth generation(6G)of mobile communication system is witnessing a new paradigm shift,i.e.,integrated sensing-communication system.A comprehensive dataset is a prerequisite for 6G integrated sensing-communication research.This paper develops a novel simulation dataset,named M3SC,for mixed multi-modal(MMM)sensing-communication integration,and the generation framework of the M3SC dataset is further given.To obtain multimodal sensory data in physical space and communication data in electromagnetic space,we utilize Air-Sim and WaveFarer to collect multi-modal sensory data and exploit Wireless InSite to collect communication data.Furthermore,the in-depth integration and precise alignment of AirSim,WaveFarer,andWireless InSite are achieved.The M3SC dataset covers various weather conditions,multiplex frequency bands,and different times of the day.Currently,the M3SC dataset contains 1500 snapshots,including 80 RGB images,160 depth maps,80 LiDAR point clouds,256 sets of mmWave waveforms with 8 radar point clouds,and 72 channel impulse response(CIR)matrices per snapshot,thus totaling 120,000 RGB images,240,000 depth maps,120,000 LiDAR point clouds,384,000 sets of mmWave waveforms with 12,000 radar point clouds,and 108,000 CIR matrices.The data processing result presents the multi-modal sensory information and communication channel statistical properties.Finally,the MMM sensing-communication application,which can be supported by the M3SC dataset,is discussed.

Multi-task Learning of Semantic Segmentation and Height Estimation for Multi-modal Remote Sensing Images

Deep learning based methods have been successfully applied to semantic segmentation of optical remote sensing images.However,as more and more remote sensing data is available,it is a new challenge to comprehensively utilize multi-modal remote sensing data to break through the performance bottleneck of single-modal interpretation.In addition,semantic segmentation and height estimation in remote sensing data are two tasks with strong correlation,but existing methods usually study individual tasks separately,which leads to high computational resource overhead.To this end,we propose a Multi-Task learning framework for Multi-Modal remote sensing images(MM_MT).Specifically,we design a Cross-Modal Feature Fusion(CMFF)method,which aggregates complementary information of different modalities to improve the accuracy of semantic segmentation and height estimation.Besides,a dual-stream multi-task learning method is introduced for Joint Semantic Segmentation and Height Estimation(JSSHE),extracting common features in a shared network to save time and resources,and then learning task-specific features in two task branches.Experimental results on the public multi-modal remote sensing image dataset Potsdam show that compared to training two tasks independently,multi-task learning saves 20%of training time and achieves competitive performance with mIoU of 83.02%for semantic segmentation and accuracy of 95.26%for height estimation.

Preparation of single atom catalysts for high sensitive gas sensing

Single atom catalysts(SACs)have garnered significant attention in the field of catalysis over the past decade due to their exceptional atom utilization efficiency and distinct physical and chemical properties.For the semiconductor-based electrical gas sensor,the core is the catalysis process of target gas molecules on the sensitive materials.In this context,the SACs offer great potential for highly sensitive and selective gas sensing,however,only some of the bubbles come to the surface.To facilitate practical applications,we present a comprehensive review of the preparation strategies for SACs,with a focus on overcoming the challenges of aggregation and low loading.Extensive research efforts have been devoted to investigating the gas sensing mechanism,exploring sensitive materials,optimizing device structures,and refining signal post-processing techniques.Finally,the challenges and future perspectives on the SACs based gas sensing are presented.

A Hand Features Based Fusion Recognition Network with Enhancing Multi-Modal Correlation

Fusing hand-based features in multi-modal biometric recognition enhances anti-spoofing capabilities.Additionally,it leverages inter-modal correlation to enhance recognition performance.Concurrently,the robustness and recognition performance of the system can be enhanced through judiciously leveraging the correlation among multimodal features.Nevertheless,two issues persist in multi-modal feature fusion recognition:Firstly,the enhancement of recognition performance in fusion recognition has not comprehensively considered the inter-modality correlations among distinct modalities.Secondly,during modal fusion,improper weight selection diminishes the salience of crucial modal features,thereby diminishing the overall recognition performance.To address these two issues,we introduce an enhanced DenseNet multimodal recognition network founded on feature-level fusion.The information from the three modalities is fused akin to RGB,and the input network augments the correlation between modes through channel correlation.Within the enhanced DenseNet network,the Efficient Channel Attention Network(ECA-Net)dynamically adjusts the weight of each channel to amplify the salience of crucial information in each modal feature.Depthwise separable convolution markedly reduces the training parameters and further enhances the feature correlation.Experimental evaluations were conducted on four multimodal databases,comprising six unimodal databases,including multispectral palmprint and palm vein databases from the Chinese Academy of Sciences.The Equal Error Rates(EER)values were 0.0149%,0.0150%,0.0099%,and 0.0050%,correspondingly.In comparison to other network methods for palmprint,palm vein,and finger vein fusion recognition,this approach substantially enhances recognition performance,rendering it suitable for high-security environments with practical applicability.The experiments in this article utilized amodest sample database comprising 200 individuals.The subsequent phase involves preparing for the extension of the method to larger databases.

Fine-Grained Ship Recognition Based on Visible and Near-Infrared Multimodal Remote Sensing Images: Dataset,Methodology and Evaluation

Fine-grained recognition of ships based on remote sensing images is crucial to safeguarding maritime rights and interests and maintaining national security.Currently,with the emergence of massive high-resolution multi-modality images,the use of multi-modality images for fine-grained recognition has become a promising technology.Fine-grained recognition of multi-modality images imposes higher requirements on the dataset samples.The key to the problem is how to extract and fuse the complementary features of multi-modality images to obtain more discriminative fusion features.The attention mechanism helps the model to pinpoint the key information in the image,resulting in a significant improvement in the model’s performance.In this paper,a dataset for fine-grained recognition of ships based on visible and near-infrared multi-modality remote sensing images has been proposed first,named Dataset for Multimodal Fine-grained Recognition of Ships(DMFGRS).It includes 1,635 pairs of visible and near-infrared remote sensing images divided into 20 categories,collated from digital orthophotos model provided by commercial remote sensing satellites.DMFGRS provides two types of annotation format files,as well as segmentation mask images corresponding to the ship targets.Then,a Multimodal Information Cross-Enhancement Network(MICE-Net)fusing features of visible and near-infrared remote sensing images,has been proposed.In the network,a dual-branch feature extraction and fusion module has been designed to obtain more expressive features.The Feature Cross Enhancement Module(FCEM)achieves the fusion enhancement of the two modal features by making the channel attention and spatial attention work cross-functionally on the feature map.A benchmark is established by evaluating state-of-the-art object recognition algorithms on DMFGRS.MICE-Net conducted experiments on DMFGRS,and the precision,recall,mAP0.5 and mAP0.5:0.95 reached 87%,77.1%,83.8%and 63.9%,respectively.Extensive experiments demonstrate that the proposed MICE-Net has more excellent performance on DMFGRS.Built on lightweight network YOLO,the model has excellent generalizability,and thus has good potential for application in real-life scenarios.

Multi-modal knowledge graph inference via media convergence and logic rule

Media convergence works by processing information from different modalities and applying them to different domains.It is difficult for the conventional knowledge graph to utilise multi-media features because the introduction of a large amount of information from other modalities reduces the effectiveness of representation learning and makes knowledge graph inference less effective.To address the issue,an inference method based on Media Convergence and Rule-guided Joint Inference model(MCRJI)has been pro-posed.The authors not only converge multi-media features of entities but also introduce logic rules to improve the accuracy and interpretability of link prediction.First,a multi-headed self-attention approach is used to obtain the attention of different media features of entities during semantic synthesis.Second,logic rules of different lengths are mined from knowledge graph to learn new entity representations.Finally,knowledge graph inference is performed based on representing entities that converge multi-media features.Numerous experimental results show that MCRJI outperforms other advanced baselines in using multi-media features and knowledge graph inference,demonstrating that MCRJI provides an excellent approach for knowledge graph inference with converged multi-media features.

Generative Multi-Modal Mutual Enhancement Video Semantic Communications

Recently,there have been significant advancements in the study of semantic communication in single-modal scenarios.However,the ability to process information in multi-modal environments remains limited.Inspired by the research and applications of natural language processing across different modalities,our goal is to accurately extract frame-level semantic information from videos and ultimately transmit high-quality videos.Specifically,we propose a deep learning-basedMulti-ModalMutual Enhancement Video Semantic Communication system,called M3E-VSC.Built upon a VectorQuantized Generative AdversarialNetwork(VQGAN),our systemaims to leverage mutual enhancement among different modalities by using text as the main carrier of transmission.With it,the semantic information can be extracted fromkey-frame images and audio of the video and performdifferential value to ensure that the extracted text conveys accurate semantic information with fewer bits,thus improving the capacity of the system.Furthermore,a multi-frame semantic detection module is designed to facilitate semantic transitions during video generation.Simulation results demonstrate that our proposed model maintains high robustness in complex noise environments,particularly in low signal-to-noise ratio conditions,significantly improving the accuracy and speed of semantic transmission in video communication by approximately 50 percent.

CrossFormer Embedding DeepLabv3+ for Remote Sensing Images Semantic Segmentation

High-resolution remote sensing image segmentation is a challenging task. In urban remote sensing, the presenceof occlusions and shadows often results in blurred or invisible object boundaries, thereby increasing the difficultyof segmentation. In this paper, an improved network with a cross-region self-attention mechanism for multi-scalefeatures based onDeepLabv3+is designed to address the difficulties of small object segmentation and blurred targetedge segmentation. First,we use CrossFormer as the backbone feature extraction network to achieve the interactionbetween large- and small-scale features, and establish self-attention associations between features at both large andsmall scales to capture global contextual feature information. Next, an improved atrous spatial pyramid poolingmodule is introduced to establish multi-scale feature maps with large- and small-scale feature associations, andattention vectors are added in the channel direction to enable adaptive adjustment of multi-scale channel features.The proposed networkmodel is validated using the PotsdamandVaihingen datasets. The experimental results showthat, compared with existing techniques, the network model designed in this paper can extract and fuse multiscaleinformation, more clearly extract edge information and small-scale information, and segment boundariesmore smoothly. Experimental results on public datasets demonstrate the superiority of ourmethod compared withseveral state-of-the-art networks.

Chaotic CS Encryption:An Efficient Image Encryption Algorithm Based on Chebyshev Chaotic System and Compressive Sensing

Images are the most important carrier of human information. Moreover, how to safely transmit digital imagesthrough public channels has become an urgent problem. In this paper, we propose a novel image encryptionalgorithm, called chaotic compressive sensing (CS) encryption (CCSE), which can not only improve the efficiencyof image transmission but also introduce the high security of the chaotic system. Specifically, the proposed CCSEcan fully leverage the advantages of the Chebyshev chaotic system and CS, enabling it to withstand various attacks,such as differential attacks, and exhibit robustness. First, we use a sparse trans-form to sparse the plaintext imageand then use theArnold transformto perturb the image pixels. After that,we elaborate aChebyshev Toeplitz chaoticsensing matrix for CCSE. By using this Toeplitz matrix, the perturbed image is compressed and sampled to reducethe transmission bandwidth and the amount of data. Finally, a bilateral diffusion operator and a chaotic encryptionoperator are used to perturb and expand the image pixels to change the pixel position and value of the compressedimage, and ultimately obtain an encrypted image. Experimental results show that our method can be resistant tovarious attacks, such as the statistical attack and noise attack, and can outperform its current competitors.

Remote sensing of quality traits in cereal and arable production systems:A review

Cereal is an essential source of calories and protein for the global population.Accurately predicting cereal quality before harvest is highly desirable in order to optimise management for farmers,grading harvest and categorised storage for enterprises,future trading prices,and policy planning.The use of remote sensing data with extensive spatial coverage demonstrates some potential in predicting crop quality traits.Many studies have also proposed models and methods for predicting such traits based on multiplatform remote sensing data.In this paper,the key quality traits that are of interest to producers and consumers are introduced.The literature related to grain quality prediction was analyzed in detail,and a review was conducted on remote sensing platforms,commonly used methods,potential gaps,and future trends in crop quality prediction.This review recommends new research directions that go beyond the traditional methods and discusses grain quality retrieval and the associated challenges from the perspective of remote sensing data.

Multidomain Correlation-Based Multidimensional CSI Tensor Generation for Device-FreeWi-Fi Sensing

Due to the fine-grained communication scenarios characterization and stability,Wi-Fi channel state information(CSI)has been increasingly applied to indoor sensing tasks recently.Although spatial variations are explicitlyreflected in CSI measurements,the representation differences caused by small contextual changes are easilysubmerged in the fluctuations of multipath effects,especially in device-free Wi-Fi sensing.Most existing datasolutions cannot fully exploit the temporal,spatial,and frequency information carried by CSI,which results ininsufficient sensing resolution for indoor scenario changes.As a result,the well-liked machine learning(ML)-based CSI sensing models still struggling with stable performance.This paper formulates a time-frequency matrixon the premise of demonstrating that the CSI has low-rank potential and then proposes a distributed factorizationalgorithm to effectively separate the stable structured information and context fluctuations in the CSI matrix.Finally,a multidimensional tensor is generated by combining the time-frequency gradients of CSI,which containsrich and fine-grained real-time contextual information.Extensive evaluations and case studies highlight thesuperiority of the proposal.

Color Image Compression and Encryption Algorithm Based on 2D Compressed Sensing and Hyperchaotic System

With the advent of the information security era,it is necessary to guarantee the privacy,accuracy,and dependable transfer of pictures.This study presents a new approach to the encryption and compression of color images.It is predicated on 2D compressed sensing(CS)and the hyperchaotic system.First,an optimized Arnold scrambling algorithm is applied to the initial color images to ensure strong security.Then,the processed images are con-currently encrypted and compressed using 2D CS.Among them,chaotic sequences replace traditional random measurement matrices to increase the system’s security.Third,the processed images are re-encrypted using a combination of permutation and diffusion algorithms.In addition,the 2D projected gradient with an embedding decryption(2DPG-ED)algorithm is used to reconstruct images.Compared with the traditional reconstruction algorithm,the 2DPG-ED algorithm can improve security and reduce computational complexity.Furthermore,it has better robustness.The experimental outcome and the performance analysis indicate that this algorithm can withstand malicious attacks and prove the method is effective.

Gel-Based Triboelectric Nanogenerators for Flexible Sensing:Principles,Properties,and Applications

The rapid development of the Internet of Things and artificial intelligence technologies has increased the need for wearable,portable,and self-powered flexible sensing devices.Triboelectric nanogenerators(TENGs)based on gel materials(with excellent conductivity,mechanical tunability,environmental adaptability,and biocompatibility)are considered an advanced approach for developing a new generation of flexible sensors.This review comprehensively summarizes the recent advances in gel-based TENGs for flexible sensors,covering their principles,properties,and applications.Based on the development requirements for flexible sensors,the working mechanism of gel-based TENGs and the characteristic advantages of gels are introduced.Design strategies for the performance optimization of hydrogel-,organogel-,and aerogel-based TENGs are systematically summarized.In addition,the applications of gel-based TENGs in human motion sensing,tactile sensing,health monitoring,environmental monitoring,human-machine interaction,and other related fields are summarized.Finally,the challenges of gel-based TENGs for flexible sensing are discussed,and feasible strategies are proposed to guide future research.

Fast compressed sensing spectral measurement with adaptive gradient multiscale resolution

We propose a fast,adaptive multiscale resolution spectral measurement method based on compressed sensing.The method can apply variable measurement resolution over the entire spectral range to reduce the measurement time by over 75%compared to a global high-resolution measurement.Mimicking the characteristics of the human retina system,the resolution distribution follows the principle of gradually decreasing.The system allows the spectral peaks of interest to be captured dynamically or to be specified a priori by a user.The system was tested by measuring single and dual spectral peaks,and the results of spectral peaks are consistent with those of global high-resolution measurements.

Remote sensing of air pollution incorporating integrated-path differential-absorption and coherent-Doppler lidar

An innovative complex lidar system deployed on an airborne rotorcraft platform for remote sensing of atmospheric pollution is proposed and demonstrated.The system incorporates integrated-path differential absorption lidar(DIAL) and coherent-doppler lidar(CDL) techniques using a dual tunable TEA CO_(2)laser in the 9—11 μm band and a 1.55 μm fiber laser.By combining the principles of differential absorption detection and pulsed coherent detection,the system enables agile and remote sensing of atmospheric pollution.Extensive static tests validate the system’s real-time detection capabilities,including the measurement of concentration-path-length product(CL),front distance,and path wind speed of air pollution plumes over long distances exceeding 4 km.Flight experiments is conducted with the helicopter.Scanning of the pollutant concentration and the wind field is carried out in an approximately 1 km slant range over scanning angle ranges from 45°to 65°,with a radial resolution of 30 m and10 s.The test results demonstrate the system’s ability to spatially map atmospheric pollution plumes and predict their motion and dispersion patterns,thereby ensuring the protection of public safety.

Sensing-Assisted Accurate and Fast Beam Management for Cellular-Connected mmWave UAV Network

Beam management,including initial access(IA)and beam tracking,is essential to the millimeter-wave Unmanned Aerial Vehicle(UAV)network.However,the conventional communicationonly and feedback-based schemes suffer a high delay and low accuracy of beam alignment,since they only enable the receiver to passively“hear”the information of the transmitter from the radio domain.This paper presents a novel sensing-assisted beam management approach,the first solution that fully utilizes the information from the visual domain to improve communication performance.We employ both integrated sensing and communication and computer vision techniques and design an extended Kalman filtering method for beam tracking and prediction.Besides,we also propose a novel dual identity association solution to distinguish multiple UAVs in dynamic environments.Real-world experiments and numerical results show that the proposed solution outperforms the conventional methods in IA delay,association accuracy,tracking error,and communication performance.

ConvNeXt-UperNet-Based Deep Learning Model for Road Extraction from High-Resolution Remote Sensing Images

When existing deep learning models are used for road extraction tasks from high-resolution images,they are easily affected by noise factors such as tree and building occlusion and complex backgrounds,resulting in incomplete road extraction and low accuracy.We propose the introduction of spatial and channel attention modules to the convolutional neural network ConvNeXt.Then,ConvNeXt is used as the backbone network,which cooperates with the perceptual analysis network UPerNet,retains the detection head of the semantic segmentation,and builds a new model ConvNeXt-UPerNet to suppress noise interference.Training on the open-source DeepGlobe and CHN6-CUG datasets and introducing the DiceLoss on the basis of CrossEntropyLoss solves the problem of positive and negative sample imbalance.Experimental results show that the new network model can achieve the following performance on the DeepGlobe dataset:79.40%for precision(Pre),97.93% for accuracy(Acc),69.28% for intersection over union(IoU),and 83.56% for mean intersection over union(MIoU).On the CHN6-CUG dataset,the model achieves the respective values of 78.17%for Pre,97.63%for Acc,65.4% for IoU,and 81.46% for MIoU.Compared with other network models,the fused ConvNeXt-UPerNet model can extract road information better when faced with the influence of noise contained in high-resolution remote sensing images.It also achieves multiscale image feature information with unified perception,ultimately improving the generalization ability of deep learning technology in extracting complex roads from high-resolution remote sensing images.

Enhancing visual security: An image encryption scheme based on parallel compressive sensing and edge detection embedding

A novel image encryption scheme based on parallel compressive sensing and edge detection embedding technology is proposed to improve visual security. Firstly, the plain image is sparsely represented using the discrete wavelet transform.Then, the coefficient matrix is scrambled and compressed to obtain a size-reduced image using the Fisher–Yates shuffle and parallel compressive sensing. Subsequently, to increase the security of the proposed algorithm, the compressed image is re-encrypted through permutation and diffusion to obtain a noise-like secret image. Finally, an adaptive embedding method based on edge detection for different carrier images is proposed to generate a visually meaningful cipher image. To improve the plaintext sensitivity of the algorithm, the counter mode is combined with the hash function to generate keys for chaotic systems. Additionally, an effective permutation method is designed to scramble the pixels of the compressed image in the re-encryption stage. The simulation results and analyses demonstrate that the proposed algorithm performs well in terms of visual security and decryption quality.

Power equipment vibration visualization using intelligent sensing method based on event-sensing principle

Vibration measurements can be used to evaluate the operation status of power equipment and are widely applied in equipment quality inspection and fault identification.Event-sensing technology can sense the change in surface light intensity caused by object vibration and provide a visual description of vibration behavior.Based on the analysis of the principle underlying the transformation of vibration behavior into event flow data by an event sensor,this paper proposes an algorithm to reconstruct event flow data into a relationship correlating vibration displacement and time to extract the amplitude-frequency characteristics of the vibration signal.A vibration measurement test platform is constructed,and feasibility and effectiveness tests are performed for the vibration motor and other power equipment.The results show that event-sensing technology can effectively perceive the surface vibration behavior of power and provide a wide dynamic range.Furthermore,the vibration measurement and visualization algorithm for power equipment constructed using this technology offers high measurement accuracy and efficiency.The results of this study provide a new noncontact and visual method for locating vibrations and performing amplitude-frequency analysis on power equipment.

Facile Semiconductor p-n Homojunction Nanowires with Strategic p-Type Doping Engineering Combined with Surface Reconstruction for Biosensing Applications

Photosensors with versatile functionalities have emerged as a cornerstone for breakthroughs in the future optoelectronic systems across a wide range of applications.In particular,emerging photoelectrochemical(PEC)-type devices have recently attracted extensive interest in liquid-based biosensing applications due to their natural electrolyte-assisted operating characteristics.Herein,a PEC-type photosensor was carefully designed and constructed by employing gallium nitride(GaN)p-n homojunction semiconductor nanowires on silicon,with the p-GaN segment strategically doped and then decorated with cobalt-nickel oxide(CoNiO_(x)).Essentially,the p-n homojunction configuration with facile p-doping engineering improves carrier separation efficiency and facilitates carrier transfer to the nanowire surface,while CoNiO_(x)decoration further boosts PEC reaction activity and carrier dynamics at the nanowire/electrolyte interface.Consequently,the constructed photosensor achieves a high responsivity of 247.8 mA W^(-1)while simultaneously exhibiting excellent operating stability.Strikingly,based on the remarkable stability and high responsivity of the device,a glucose sensing system was established with a demonstration of glucose level determination in real human serum.This work offers a feasible and universal approach in the pursuit of high-performance bio-related sensing applications via a rational design of PEC devices in the form of nanostructured architecture with strategic doping engineering.