Towards complex scenes: A deep learning-based camouflaged people detection method for snapshot multispectral images

Camouflaged people are extremely expert in actively concealing themselves by effectively utilizing cover and the surrounding environment. Despite advancements in optical detection capabilities through imaging systems, including spectral, polarization, and infrared technologies, there is still a lack of effective real-time method for accurately detecting small-size and high-efficient camouflaged people in complex real-world scenes. Here, this study proposes a snapshot multispectral image-based camouflaged detection model, multispectral YOLO(MS-YOLO), which utilizes the SPD-Conv and Sim AM modules to effectively represent targets and suppress background interference by exploiting the spatial-spectral target information. Besides, the study constructs the first real-shot multispectral camouflaged people dataset(MSCPD), which encompasses diverse scenes, target scales, and attitudes. To minimize information redundancy, MS-YOLO selects an optimal subset of 12 bands with strong feature representation and minimal inter-band correlation as input. Through experiments on the MSCPD, MS-YOLO achieves a mean Average Precision of 94.31% and real-time detection at 65 frames per second, which confirms the effectiveness and efficiency of our method in detecting camouflaged people in various typical desert and forest scenes. Our approach offers valuable support to improve the perception capabilities of unmanned aerial vehicles in detecting enemy forces and rescuing personnel in battlefield.

Modeling and analysis for the image mapping spectrometer

The snapshot image mapping spectrometer（IMS） has advantages such as high temporal resolution,high throughput,compact structure and simple reconstructed algorithm.In recent years,it has been utilized in biomedicine,remote sensing,etc.However,the system errors and various factors can cause cross talk,image degradation and spectral distortion in the system.In this research,a theoretical model is presented along with the point response function（PRF） for the IMS,and the influence of the mirror tilt angle error of the image mapper and the prism apex angle error are analyzed based on the model.The results indicate that the tilt angle error causes loss of light throughput and the prism apex angle error causes spectral mixing between adjacent sub-images.The light intensity on the image plane is reduced to 95%when the mirror tilt angle error is increased to ±100 ＂（≈ 0.028°）.The prism apex error should be controlled within the range of 0-36＂（0.01°）to ensure the designed number of spectral bands,and avoid spectral mixing between adjacent images.

In-situ real-time monitoring of ultrafast laser processing using wide-field high-resolution snapshot compressive microscopy

Over the last few decades,ultrafast laser processing has become a widely used tool for manufacturing microstructures and nanostructures.The real-time monitoring of laser material processing provides opportunities to inspect processes and provide feedback.To date,in-situ and real-time monitoring of laser material processing has rarely been performed.To this end,we propose dual-path snapshot compressive microscopy(DP-SCM)for high-speed,large field-of-view,and high-resolution imaging for in-situ and real-time ultrafast laser processing.In the evaluation of DP-SCM,the field of view,lateral resolution,and imaging speed were measured to be 2 mm,775 nm,and 500 fps,respectively.In ultrafast laser processing,the laser scanning process is observed using a DP-SCM system when translating the sample stage and scanning the focused femtosecond laser.Finally,we monitored the development of a self-organized nanograting structure to validate the potential of our system for unveiling new material mechanisms.The proposed method serves as an add-up(plug-and-play)module for any imaging setup and has vast potential for opening new avenues for high-throughput imaging in laser material processing.

From compressive sampling to compressive tasking:retrieving semantics in compressed domain with low bandwidth

High-throughput imaging is highly desirable in intelligent analysis of computer vision tasks.In conventional design,throughput is limited by the separation between physical image capture and digital post processing.Computational imaging increases throughput by mixing analog and digital processing through the image capture pipeline.Yet,recent advances of computational imaging focus on the“compressive sampling”,this precludes the wide applications in practical tasks.This paper presents a systematic analysis of the next step for computational imaging built on snapshot compressive imaging(SCI)and semantic computer vision(SCV)tasks,which have independently emerged over the past decade as basic computational imaging platforms.SCI is a physical layer process that maximizes information capacity per sample while minimizing system size,power and cost.SCV is an abstraction layer process that analyzes image data as objects and features,rather than simple pixel maps.In current practice,SCI and SCV are independent and sequential.This concatenated pipeline results in the following problems:i)a large amount of resources are spent on task-irrelevant computation and transmission,ii)the sampling and design efficiency of SCI is attenuated,and iii)the final performance of SCV is limited by the reconstruction errors of SCI.Bearing these concerns in mind,this paper takes one step further aiming to bridge the gap between SCI and SCV to take full advantage of both approaches.After reviewing the current status of SCI,we propose a novel joint framework by conducting SCV on raw measurements captured by SCI to select the region of interest,and then perform reconstruction on these regions to speed up processing time.We use our recently built SCI prototype to verify the framework.Preliminary results are presented and the prospects for a joint SCI and SCV regime are discussed.By conducting computer vision tasks in the compressed domain,we envision that a new era of snapshot compressive imaging with limited end-to-end bandwidth is coming.