Real-time and high-transmission middle-infrared optical imaging system based on a pixel-wise metasurface micro-polarization array

Real-time polarization medium-wave infrared(MIR)optical imaging systems enable the acquisition of infrared and polarization information for a target.At present,real-time polarization MIR devices face the following problems:poor real-time performance,low transmission and high requirements for fabrication and integration.Herein,we aim to improve the performance of real-time polarization imaging systems in the MIR waveband and solve the above-mentioned defects.Therefore,we propose a MIR polarization imaging system to achieve real-time polarization-modulated imaging with high transmission as well as improved performance based on a pixel-wise metasurface micro-polarization array(PMMPA).The PMMPA element comprises several linear polarization(LP)filters with different polarization angles.The optimization results demonstrate that the transmittance of the center field of view for the LP filters is up to 77%at a wavelength of4.0μm and an extinction ratio of 88 d B.In addition,a near-diffraction-limited real-time MIR imaging optical system is designed with a field of view of 5°and an F-number of 2.The simulation results show that an MIR polarization imaging system with excellent real-time performance and high transmission is achieved by using the optimized PMMPA element.Therefore,the method is compatible with the available optical system design technologies and provides a way to realize real-time polarization imaging in MIR wavebands.

A Flexible DSP-Based Network forReal-Time Image-Processing

This paper proposed a general purpose real-time image processing system based on a flexible DSP-based Network, which is implemented by a high bandwidth communication channel, links. The links is realized using FPGA and provides a bandwidth of 12. 8 Gbit/s. Using the links, The topologic of multi-DSP system can be changed online to meet the variabilities of the parallel algorithm of image processing. The system can be assembled with utmost tens of boards and maintain the high communication speed. Analysis of the system adaptivity to image processing is testified followed by actual results. Key words real-time image processing - multi-DSP - flexible - scalable - FPGA - links CLC number TP 303 Foundation item: Supported by the National Natural Science Foundation of China (60135020)Biography: MAO Hai-cen(1973-), male, Ph.D. candidate, research direction: artificial intelligence, expert system, pattern recognition and image processing

New multi-DSP parallel computing architecture for real-time image processing

The flexibility of traditional image processing system is limited because those system are designed for specific applications. In this paper, a new TMS320C64x-based multi-DSP parallel computing architecture is presented. It has many promising characteristics such as powerful computing capability, broad I/O bandwidth, topology flexibility, and expansibility. The parallel system performance is evaluated by practical experiment.

Image completion algorithm based on texture synthesis

A new algorithm is proposed for completing the missing parts caused by the removal of foreground or background elements from an image of natural scenery in a visually plausible way. The major contributions of the proposed algorithm are： （1） for most natural images, there is a strong orientation of texture or color distribution. So a method is introduced to compute the main direction of the texture and complete the image by limiting the search to one direction to carry out image completion quite fast; （2） there exists a synthesis ordering for image completion. The searching order of the patches is defined to ensure the regions with more known information and the structures should be completed before filling in other regions; （3） to improve the visual effect of texture synthesis, an adaptive scheme is presented to determine the size of the template window for capturing the features of various scales. A number of examples are given to demonstrate the effectiveness of the proposed algorithm.

DELAUNAY TRIANGULATION AND IMAGE DENSE MATCHING IN VIEW SYNTHESIS

A new method of view synthesis is proposed based on Delaunay triangulation. The first step of this method is making the Delaunay triangulation of 2 reference images. Secondly, matching the image points using the epipolar geometry constraint. Finally, constructing the third view according to pixel transferring under the trilinear constraint. The method gets rid of the classic time consuming dense matching technique and takes advantage of Delaunay triangulation. So it can not only save the computation time but also enhance the quality of the synthesized view. The significance of this method is that it can be used directly in the fields of video coding, image compressing and virtual reality.

Three-Dimensional Space Interpolation of Grey / Depth Image Sequence-A New Technique of Computer Graphics Synthesis

This paper advances a three-dimensional space interpolation method of grey / depth image sequence, which breaks free from the limit of original practical photographing route. Pictures can cruise at will in space. By using space sparse sampling, great memorial capacity can be saved and reproduced scenes can be controlled. To solve time consuming and complex computations in three-dimensional interpolation algorithm, we have studied a fast and practical algorithm of scattered space lattice and that of 'Warp' algorithm with proper depth. By several simple aspects of three dimensional space interpolation, we succeed in developing some simple and practical algorithms. Some results of simulated experiments with computers have shown that the new method is absolutely feasible.

High performance reconfigurable hardware system for real-time image processing

A novel reconfigurable hardware system which uses both muhi-DSP and FPGA to attain high performance and real-time image processing are presented. The system structure and working principle of mainly processing multi-BSP board, extended multi-DSP board are analysed. The outstanding advantage is that the communication among different board components of this system is supported by high speed link ports ＆ serial ports for increasing the system performance and computational power. Then the implementation of embedded real-time operating systems （RTOS） by us is discussed in detail. In this system, we adopt two kinds of parallel structures controlled by RTOS for parallel processing of algorithms. The experimental results show that exploitive period of the system is short, and maintenance convenient. Thus it is suitable for real-time image processing and can get satisfactory effect of image recognition.

An FPGA Design for Real-Time Image Denoising

The increasing use of images in miscellaneous applications such as medical image analysis and visual quality inspection has led to growing interest in image processing.However,images are often contaminated with noise which may corrupt any of the following image processing steps.Therefore,noise filtering is often a necessary preprocessing step for the most image processing applications.Thus,in this paper an optimized field-programmable gate array(FPGA)design is proposed to implement the adaptive vector directional distance filter(AVDDF)in hardware/software(HW/SW)codesign context for removing noise from the images in real-time.For that,the high-level synthesis(HLS)flow is used through the Xilinx Vivado HLS tool to reduce the design complexity of the HW part.The SW part is developed based on C/C++programming language and executed on an advanced reduced instruction set computer(RISC)machines(ARM)Cortex-A53 processor.The communication between the SW and HW parts is achieved using the advanced extensible Interface stream(AXI-stream)interface to increase the data bandwidth.The experiment results on the Xilinx ZCU102 FPGA board show an improvement in processing time of the AVDDF filter by 98%for the HW/SW implementation relative to the SW implementation.This result is given for the same quality of image between the HW/SW and SW implementations in terms of the normalized color difference(NCD)and the peak signal to noise ratio(PSNR).

An FPGA-based real-time image processing system

This paper analyzes the current difficulties encountered in on-line inspection systems of strip surface quality, specifically relating to problems with real-time processing of huge amounts of data. To address this need, this paper describes an FPGA-based high-speed image processing module with both hardware and software aspects. Improving these two aspects together will help the system achieve real-time processing of massive image data, and simplifies the architecture of the strip surface quality on-line inspection system.

Image Fusion Real-time System Based on FPGA and Multi-DSP

In order to solve complex algorithm that is difficult to achieve real-time processing of Multiband image fusion within large amount of data, a real-time image fusion system based on FPGA and multi-DSP is designed. Five-band image acquisition, image registration, image fusion and display output can be done within the system which uses FPGA as the main processor and the other three DSP as an algorithm processor. Making full use of Flexible and high-speed characteristics of FPGA, while an image fusion algorithm based on multi-wavelet transform is optimized and applied to the system. The final experimental results show that the frame rate of 15 Hz, with a resolution of 1392 × 1040 of the five-band image can be used by the system to complete processing within 41ms.

A Survey of GAN Based Image Synthesis

Image generation is a hot topic in the academic recently,and has been applied to AI drawing,which can bring Vivid AI paintings without labor costs.In image generation,we represent the image as a random vector,assuming that the images of the natural scene obey an unknown distribution,we hope to estimate its distribution through some observation samples.Especially,with the development of GAN(Generative Adversarial Network),The generator and discriminator improve the model capability through adversarial,the quality of the generated image is also increasing.The image quality generated by the existing GAN based image generation model is so well-paint that it can be passed for genuine one.Based on the brief introduction of the concept ofGAN,this paper analyzes themain ideas of image synthesis,studies the representative SOTA GAN based Image synthesis method.

View synthesis based on the serial images from camera lengthways motion

For the pre-acquired serial images from camera lengthways motion, a view synthesis algorithm based on epipolar geometry constraint is proposed in this paper. It uses the whole matching and maintaining order characters of the epipolar line, Fourier transform and dynamic programming matching theories, thus truly synthesizing the destination image of current viewpoint. Through the combination of Fourier transform, epipolar geometry constraint and dynamic programming matching, the circumference distortion problem resulting from conventional view synthesis approaches is effectively avoided. The detailed implementation steps of this algorithm are given, and some running instances are presented to illustrate the results.

A fast and adaptive method for automatic weld defect detection in various real-time X-ray imaging systems

A first and effective method is proposed to detect weld deject adaptively in various Dypes of real-time X-ray images obtained in different conditions. After weld extraction and noise reduction, a proper template of median filter is used to estimate the weld background. After the weld background is subtracted from the original image, an adaptite threshold segmentation algorithm is proposed to obtain the binary image, and then the morphological close and open operation, labeling algorithm and fids＇e alarm eliminating algorithm are applied to pracess the binary image to obtain the defect, ct detection result. At last, a fast realization procedure jbr proposed method is developed. The proposed method is tested in real-time X-ray image,s obtairted in different X-ray imaging sutems. Experiment results show that the proposed method is effective to detect low contrast weld dejects with few .false alarms and is adaptive to various types of real-time X-ray imaging systems.

A new imaging mode based on X-ray CT as prior image and sparsely sampled projections for rapid clinical proton CT

Proton computed tomography(CT)has a distinct practical significance in clinical applications.It eliminates 3–5%errors caused by the transformation of Hounsfield unit(HU)to relative stopping power(RSP)values when using X-ray CT for positioning and treatment planning systems(TPSs).Following the development of FLASH proton therapy,there are increased requirements for accurate and rapid positioning in TPSs.Thus,a new rapid proton CT imaging mode is proposed based on sparsely sampled projections.The proton beam was boosted to 350 MeV by a compact proton linear accelerator(LINAC).In this study,the comparisons of the proton scattering with the energy of 350 MeV and 230 MeV are conducted based on GEANT4 simulations.As the sparsely sampled information associated with beam acquisitions at 12 angles is not enough for reconstruction,X-ray CT is used as a prior image.The RSP map generated by converting the X-ray CT was constructed based on Monte Carlo simulations.Considering the estimation of the most likely path(MLP),the prior image-constrained compressed sensing(PICCS)algorithm is used to reconstruct images from two different phantoms using sparse proton projections of 350 MeV parallel proton beam.The results show that it is feasible to realize the proton image reconstruction with the rapid proton CT imaging proposed in this paper.It can produce RSP maps with much higher accuracy for TPSs and fast positioning to achieve ultra-fast imaging for real-time image-guided radiotherapy(IGRT)in clinical proton therapy applications.

Heterogeneity induced strain localization in block-in-matrix-soils subjected to uniaxial loading using real-time CT scanning

Block-in-matrix-soils(bimsoils)are geological mixtures that have distinct structures consisting of relatively strong rock blocks and weak matrix soils.It is still a challenge to evaluate the mechanical behaviors of bimsoils because of the heterogeneity,chaotic structure,and lithological variability.As a result,only very limited laboratory studies have been reported on the evolution of their internal deformation.In this study,the deformation evolution of bimsoils under uniaxial loading is investigated using real-time X-ray computed tomography(CT)and image correlation algorithm(with a rock block percentage(RBP)of 40%).Three parameters,i.e.heterogeneity coefficient(K),correlation coefficient(CC),and standard deviation(STD)of displacement fields,are proposed to quantify the heterogeneity of the motion of the rock blocks and the progressive deformation of the bimsoils.Experimental results show that the rock blocks in bimsoils are prone to forming clusters with increasing loading,and the sliding surface goes around only one side of a cluster.Based on the movement of the rock blocks recorded by STD and CC,the progressive deformation of the bimsoils is quantitatively divided into three stages:initialization of the rotation of rock blocks,formation of rock block clusters,and formation of a shear band by rock blocks with significant rotation.Moreover,the experimental results demonstrate that the meso-motion of rock blocks controls the macroscopic mechanical properties of the samples.

A Novel Unsupervised MRI Synthetic CT Image Generation Framework with Registration Network

In recent years,radiotherapy based only on Magnetic Resonance(MR)images has become a hot spot for radiotherapy planning research in the current medical field.However,functional computed tomography(CT)is still needed for dose calculation in the clinic.Recent deep-learning approaches to synthesized CT images from MR images have raised much research interest,making radiotherapy based only on MR images possible.In this paper,we proposed a novel unsupervised image synthesis framework with registration networks.This paper aims to enforce the constraints between the reconstructed image and the input image by registering the reconstructed image with the input image and registering the cycle-consistent image with the input image.Furthermore,this paper added ConvNeXt blocks to the network and used large kernel convolutional layers to improve the network’s ability to extract features.This research used the collected head and neck data of 180 patients with nasopharyngeal carcinoma to experiment and evaluate the training model with four evaluation metrics.At the same time,this research made a quantitative comparison of several commonly used model frameworks.We evaluate the model performance in four evaluation metrics which achieve Mean Absolute Error(MAE),Root Mean Square Error(RMSE),Peak Signal-to-Noise Ratio(PSNR),and Structural Similarity(SSIM)are 18.55±1.44,86.91±4.31,33.45±0.74 and 0.960±0.005,respectively.Compared with other methods,MAE decreased by 2.17,RMSE decreased by 7.82,PSNR increased by 0.76,and SSIM increased by 0.011.The results show that the model proposed in this paper outperforms other methods in the quality of image synthesis.The work in this paper is of guiding significance to the study of MR-only radiotherapy planning.

Tight Sandstone Image Augmentation for Image Identification Using Deep Learning

Intelligent identification of sandstone slice images using deep learning technology is the development trend of mineral identification,and accurate mineral particle segmentation is the most critical step for intelligent identification.A typical identification model requires many training samples to learn as many distinguishable features as possible.However,limited by the difficulty of data acquisition,the high cost of labeling,and privacy protection,this has led to a sparse sample number and cannot meet the training requirements of deep learning image identification models.In order to increase the number of samples and improve the training effect of deep learning models,this paper proposes a tight sandstone image data augmentation method by combining the advantages of the data deformation method and the data oversampling method in the Putaohua reservoir in the Sanzhao Sag of the Songliao Basin as the target area.First,the Style Generative Adversarial Network(StyleGAN)is improved to generate high-resolution tight sandstone images to improve data diversity.Second,we improve the Automatic Data Augmentation(AutoAugment)algorithm to search for the optimal augmentation strategy to expand the data scale.Finally,we design comparison experiments to demonstrate that this method has obvious advantages in generating image quality and improving the identification effect of deep learning models in real application scenarios.

Real-time Three-Dimensional Color Doppler Flow Imaging： An Improved Technique for Quantitative Analysis of Aortic Regurgitation

The recently introduced real-time three-dimensional color Doppler flow imaging （RT-3D CDFI） technique provides a quick and accurate calculation of regurgitant jet volume （RJV） and fraction. In order to evaluate RT-3D CDFI in the noninvasive assessment of aortic RJV and regurgitant jet fraction （RJF） in patients with isolated aortic regurgitation, real-time three-dimensional echocardiographic studies were performed on 23 patients with isolated aortic regurgitation to obtain LV end-diastolic volumes （LVEDV）, end-systolic volumes （LVESV） and RJV, and then RJF could be calculated. The regurgitant volume （RV） and regurgitant fraction （RF） calculated by two-dimensional pulsed Doppler （2D-PD） method served as reference values. The results showed that aortic RJV measured by the RT-3D CDFI method showed a good correlation with the 2D-PD measurements （r= 0.93, Y=0.89X＋ 3.9, SEE= 8.6 mL, P〈0.001 ）; the mean （SD） difference between the two methods was - 1.5 （9.8） mL. % RJF estimated by the RT-3D CDFI method was also correlated well with the values obtained by the 2D-PD method （r=0.88, Y=0.71X＋ 14.8, SEE= 6.4 %, P〈0. 001）; the mean （SD） difference between the two methods was -1.2 （7.9） %. It was suggested that the newly developed RT-3D CDFI technique was feasible in the majority of patients. In patients with eccentric aortic regurgitation, this new modality provides additional information to that obtained from the two-dimensional examination, which overcomes the inherent limitations of two-dimensional echocardiography by depicting the full extent of the jet trajectory. In addition, the RT-3D CDFI method is quick and accurate in calculating RJV and RJF.

An Intelligent Detection Method for Optical Remote Sensing Images Based on Improved YOLOv7

To address the issue of imbalanced detection performance and detection speed in current mainstream object detection algorithms for optical remote sensing images,this paper proposes a multi-scale object detection model for remote sensing images on complex backgrounds,called DI-YOLO,based on You Only Look Once v7-tiny(YOLOv7-tiny).Firstly,to enhance the model’s ability to capture irregular-shaped objects and deformation features,as well as to extract high-level semantic information,deformable convolutions are used to replace standard convolutions in the original model.Secondly,a Content Coordination Attention Feature Pyramid Network(CCA-FPN)structure is designed to replace the Neck part of the original model,which can further perceive relationships between different pixels,reduce feature loss in remote sensing images,and improve the overall model’s ability to detect multi-scale objects.Thirdly,an Implicitly Efficient Decoupled Head(IEDH)is proposed to increase the model’s flexibility,making it more adaptable to complex detection tasks in various scenarios.Finally,the Smoothed Intersection over Union(SIoU)loss function replaces the Complete Intersection over Union(CIoU)loss function in the original model,resulting in more accurate prediction of bounding boxes and continuous model optimization.Experimental results on the High-Resolution Remote Sensing Detection(HRRSD)dataset demonstrate that the proposed DI-YOLO model outperforms mainstream target detection algorithms in terms of mean Average Precision(mAP)for optical remote sensing image detection.Furthermore,it achieves Frames Per Second(FPS)of 138.9,meeting fast and accurate detection requirements.