Real-time image processing and display in object size detection based on VC++

Real-time detection for object size has now become a hot topic in the testing field and image processing is the core algorithm. This paper focuses on the processing and display of the collected dynamic images to achieve a real-time image pro- cessing for the moving objects. Firstly, the median filtering, gain calibration, image segmentation, image binarization, cor- ner detection and edge fitting are employed to process the images of the moving objects to make the image close to the real object. Then, the processed images are simultaneously displayed on a real-time basis to make it easier to analyze, understand and identify them, and thus it reduces the computation complexity. Finally, human-computer interaction （HCI）-friendly in- terface based on VC ＋＋ is designed to accomplish the digital logic transform, image processing and real-time display of the objects. The experiment shows that the proposed algorithm and software design have better real-time performance and accu- racy which can meet the industrial needs.

A Flexible DSP-Based Network forReal-Time Image-Processing

This paper proposed a general purpose real-time image processing system based on a flexible DSP-based Network, which is implemented by a high bandwidth communication channel, links. The links is realized using FPGA and provides a bandwidth of 12. 8 Gbit/s. Using the links, The topologic of multi-DSP system can be changed online to meet the variabilities of the parallel algorithm of image processing. The system can be assembled with utmost tens of boards and maintain the high communication speed. Analysis of the system adaptivity to image processing is testified followed by actual results. Key words real-time image processing - multi-DSP - flexible - scalable - FPGA - links CLC number TP 303 Foundation item: Supported by the National Natural Science Foundation of China (60135020)Biography: MAO Hai-cen(1973-), male, Ph.D. candidate, research direction: artificial intelligence, expert system, pattern recognition and image processing

High performance reconfigurable hardware system for real-time image processing

A novel reconfigurable hardware system which uses both muhi-DSP and FPGA to attain high performance and real-time image processing are presented. The system structure and working principle of mainly processing multi-BSP board, extended multi-DSP board are analysed. The outstanding advantage is that the communication among different board components of this system is supported by high speed link ports ＆ serial ports for increasing the system performance and computational power. Then the implementation of embedded real-time operating systems （RTOS） by us is discussed in detail. In this system, we adopt two kinds of parallel structures controlled by RTOS for parallel processing of algorithms. The experimental results show that exploitive period of the system is short, and maintenance convenient. Thus it is suitable for real-time image processing and can get satisfactory effect of image recognition.

An FPGA-based real-time image processing system

This paper analyzes the current difficulties encountered in on-line inspection systems of strip surface quality, specifically relating to problems with real-time processing of huge amounts of data. To address this need, this paper describes an FPGA-based high-speed image processing module with both hardware and software aspects. Improving these two aspects together will help the system achieve real-time processing of massive image data, and simplifies the architecture of the strip surface quality on-line inspection system.

APPLICATION AND IMPLEMENTATION OF REAL-TIME IMAGE ZOOMING WITH THE WALSH TECHNIQUE

Based on the study of Walsh transformation,the zooming template of a two dimensional superimposure filter is decomposed and simplified,and it is real time implemented with FPGA.This method is simple and effective.The quality of the image is very good.

New multi-DSP parallel computing architecture for real-time image processing

The flexibility of traditional image processing system is limited because those system are designed for specific applications. In this paper, a new TMS320C64x-based multi-DSP parallel computing architecture is presented. It has many promising characteristics such as powerful computing capability, broad I/O bandwidth, topology flexibility, and expansibility. The parallel system performance is evaluated by practical experiment.

An FPGA Design for Real-Time Image Denoising

The increasing use of images in miscellaneous applications such as medical image analysis and visual quality inspection has led to growing interest in image processing.However,images are often contaminated with noise which may corrupt any of the following image processing steps.Therefore,noise filtering is often a necessary preprocessing step for the most image processing applications.Thus,in this paper an optimized field-programmable gate array(FPGA)design is proposed to implement the adaptive vector directional distance filter(AVDDF)in hardware/software(HW/SW)codesign context for removing noise from the images in real-time.For that,the high-level synthesis(HLS)flow is used through the Xilinx Vivado HLS tool to reduce the design complexity of the HW part.The SW part is developed based on C/C++programming language and executed on an advanced reduced instruction set computer(RISC)machines(ARM)Cortex-A53 processor.The communication between the SW and HW parts is achieved using the advanced extensible Interface stream(AXI-stream)interface to increase the data bandwidth.The experiment results on the Xilinx ZCU102 FPGA board show an improvement in processing time of the AVDDF filter by 98%for the HW/SW implementation relative to the SW implementation.This result is given for the same quality of image between the HW/SW and SW implementations in terms of the normalized color difference(NCD)and the peak signal to noise ratio(PSNR).

Real-time continuous image guidance for endoscopic retrograde cholangiopancreatography based on 3D/2D registration and respiratory compensation

BACKGROUND It has been confirmed that three-dimensional(3D)imaging allows easier identification of bile duct anatomy and intraoperative guidance of endoscopic retrograde cholangiopancreatography(ERCP),which reduces the radiation dose and procedure time with improved safety.However,current 3D biliary imaging does not have good real-time fusion with intraoperative imaging,a process meant to overcome the influence of intraoperative respiratory motion and guide navigation.The present study explored the feasibility of real-time continuous image-guided ERCP.AIM To explore the feasibility of real-time continuous image-guided ERCP.METHODS We selected 23D-printed abdominal biliary tract models with different structures to simulate different patients.The ERCP environment was simulated for the biliary phantom experiment to create a navigation system,which was further tested in patients.In addition,based on the estimation of the patient’s respiratory motion,preoperative 3D biliary imaging from computed tomography of 18 patients with cholelithiasis was registered and fused in real-time with 2D fluoroscopic sequence generated by the C-arm unit during ERCP.RESULTS Continuous image-guided ERCP was applied in the biliary phantom with a registration error of 0.46 mm±0.13 mm and a tracking error of 0.64 mm±0.24mm.After estimating the respiratory motion,3D/2D registration accurately transformed preoperative 3D biliary images to each image in the X-ray image sequence in real-time in 18 patients,with an average fusion rate of 88%.CONCLUSION Continuous image-guided ERCP may be an effective approach to assist the operator and reduce the use of X-ray and contrast agents.

A Practical Study of Intelligent Image-Based Mobile Robot for Tracking Colored Objects

Object tracking is one of the major tasks for mobile robots in many real-world applications.Also,artificial intelligence and automatic control techniques play an important role in enhancing the performance of mobile robot navigation.In contrast to previous simulation studies,this paper presents a new intelligent mobile robot for accomplishing multi-tasks by tracking red-green-blue(RGB)colored objects in a real experimental field.Moreover,a practical smart controller is developed based on adaptive fuzzy logic and custom proportional-integral-derivative(PID)schemes to achieve accurate tracking results,considering robot command delay and tolerance errors.The design of developed controllers implies some motion rules to mimic the knowledge of experienced operators.Twelve scenarios of three colored object combinations have been successfully tested and evaluated by using the developed controlled image-based robot tracker.Classical PID control failed to handle some tracking scenarios in this study.The proposed adaptive fuzzy PID control achieved the best accurate results with the minimum average final error of 13.8 cm to reach the colored targets,while our designed custom PID control is efficient in saving both average time and traveling distance of 6.6 s and 14.3 cm,respectively.These promising results demonstrate the feasibility of applying our developed image-based robotic system in a colored object-tracking environment to reduce human workloads.

Improving Yolo5 for Real-Time Detection of Small Targets in Side Scan Sonar Images

Side scan sonar(SSS)is an important means to detect and locate seafloor targets.Autonomous underwater vehicles(AUVs)carrying SSS stay near the seafloor to obtain high-resolution images and provide the outline of the target for observers.The target feature information of an SSS image is similar to the background information,and a small target has less pixel information;therefore,accu-rately identifying and locating small targets in SSS images is challenging.We collect the SSS images of iron metal balls(with a diameter of 1m)and rocks to solve the problem of target misclassification.Thus,the dataset contains two types of targets,namely,‘ball’and‘rock’.With the aim to enable AUVs to accurately and automatically identify small underwater targets in SSS images,this study designs a multisize parallel convolution module embedded in state-of-the-art Yolo5.An attention mechanism transformer and a convolutional block attention module are also introduced to compare their contributions to small target detection accuracy.The performance of the proposed method is further evaluated by taking the lightweight networks Mobilenet3 and Shufflenet2 as the backbone network of Yolo5.This study focuses on the performance of convolutional neural networks for the detection of small targets in SSS images,while another comparison experiment is carried out using traditional HOG+SVM to highlight the neural network’s ability.This study aims to improve the detection accuracy while ensuring the model efficiency to meet the real-time working requirements of AUV target detection.

Dendritic Learning-Incorporated Vision Transformer for Image Recognition

Dear Editor,This letter proposes to integrate dendritic learnable network architecture with Vision Transformer to improve the accuracy of image recognition.In this study,based on the theory of dendritic neurons in neuroscience,we design a network that is more practical for engineering to classify visual features.Based on this,we propose a dendritic learning-incorporated vision Transformer(DVT),which out-performs other state-of-the-art methods on three image recognition benchmarks.

Infrared and Visible Image Fusion Based on Res2Net-Transformer Automatic Encoding and Decoding

A novel image fusion network framework with an autonomous encoder and decoder is suggested to increase thevisual impression of fused images by improving the quality of infrared and visible light picture fusion. The networkcomprises an encoder module, fusion layer, decoder module, and edge improvementmodule. The encoder moduleutilizes an enhanced Inception module for shallow feature extraction, then combines Res2Net and Transformerto achieve deep-level co-extraction of local and global features from the original picture. An edge enhancementmodule (EEM) is created to extract significant edge features. A modal maximum difference fusion strategy isintroduced to enhance the adaptive representation of information in various regions of the source image, therebyenhancing the contrast of the fused image. The encoder and the EEM module extract features, which are thencombined in the fusion layer to create a fused picture using the decoder. Three datasets were chosen to test thealgorithmproposed in this paper. The results of the experiments demonstrate that the network effectively preservesbackground and detail information in both infrared and visible images, yielding superior outcomes in subjectiveand objective evaluations.

Using restored two-dimensional X-ray images to reconstruct the three-dimensional magnetopause

Astronomical imaging technologies are basic tools for the exploration of the universe,providing basic data for the research of astronomy and space physics.The Soft X-ray Imager(SXI)carried by the Solar wind Magnetosphere Ionosphere Link Explorer(SMILE)aims to capture two-dimensional(2-D)images of the Earth’s magnetosheath by using soft X-ray imaging.However,the observed 2-D images are affected by many noise factors,destroying the contained information,which is not conducive to the subsequent reconstruction of the three-dimensional(3-D)structure of the magnetopause.The analysis of SXI-simulated observation images shows that such damage cannot be evaluated with traditional restoration models.This makes it difficult to establish the mapping relationship between SXIsimulated observation images and target images by using mathematical models.We propose an image restoration algorithm for SXIsimulated observation images that can recover large-scale structure information on the magnetosphere.The idea is to train a patch estimator by selecting noise–clean patch pairs with the same distribution through the Classification–Expectation Maximization algorithm to achieve the restoration estimation of the SXI-simulated observation image,whose mapping relationship with the target image is established by the patch estimator.The Classification–Expectation Maximization algorithm is used to select multiple patch clusters with the same distribution and then train different patch estimators so as to improve the accuracy of the estimator.Experimental results showed that our image restoration algorithm is superior to other classical image restoration algorithms in the SXI-simulated observation image restoration task,according to the peak signal-to-noise ratio and structural similarity.The restoration results of SXI-simulated observation images are used in the tangent fitting approach and the computed tomography approach toward magnetospheric reconstruction techniques,significantly improving the reconstruction results.Hence,the proposed technology may be feasible for processing SXI-simulated observation images.

The Soft X-ray Imager(SXI)on the SMILE Mission

The Soft X-ray Imager(SXI)is part of the scientific payload of the Solar wind Magnetosphere Ionosphere Link Explorer(SMILE)mission.SMILE is a joint science mission between the European Space Agency(ESA)and the Chinese Academy of Sciences(CAS)and is due for launch in 2025.SXI is a compact X-ray telescope with a wide field-of-view(FOV)capable of encompassing large portions of Earth’s magnetosphere from the vantage point of the SMILE orbit.SXI is sensitive to the soft X-rays produced by the Solar Wind Charge eXchange(SWCX)process produced when heavy ions of solar wind origin interact with neutral particles in Earth’s exosphere.SWCX provides a mechanism for boundary detection within the magnetosphere,such as the position of Earth’s magnetopause,because the solar wind heavy ions have a very low density in regions of closed magnetic field lines.The sensitivity of the SXI is such that it can potentially track movements of the magnetopause on timescales of a few minutes and the orbit of SMILE will enable such movements to be tracked for segments lasting many hours.SXI is led by the University of Leicester in the United Kingdom(UK)with collaborating organisations on hardware,software and science support within the UK,Europe,China and the United States.

基于Real-time PCR法检测乳粉中牛源性成分定量研究

基于Real-timePCR建立了乳粉中牛源性成分相对定量检测方法,并对牛的特异性引物与探针进行了特异性、灵敏度和稳定性测试。通过模拟不同浓度牛乳粉与马乳粉混合样本,根据其△Ct值的函数关系进行线性拟合进而绘制标准曲线,建立乳粉中牛源性成分的相对定量检测。结果显示,该方法的最低检测限为0.00001 mg/mL,回收率为91.11%~119.2%,组间变异系数≤0.58%、组内变异系数≤1.44%。说明该方法在特异性与稳定性上适用于乳粉中牛源性成分及含量的掺假检测。

Background removal from global auroral images:Data-driven dayglow modeling

Global images of auroras obtained by cameras on spacecraft are a key tool for studying the near-Earth environment.However,the cameras are sensitive not only to auroral emissions produced by precipitating particles,but also to dayglow emissions produced by photoelectrons induced by sunlight.Nightglow emissions and scattered sunlight can contribute to the background signal.To fully utilize such images in space science,background contamination must be removed to isolate the auroral signal.Here we outline a data-driven approach to modeling the background intensity in multiple images by formulating linear inverse problems based on B-splines and spherical harmonics.The approach is robust,flexible,and iteratively deselects outliers,such as auroral emissions.The final model is smooth across the terminator and accounts for slow temporal variations and large-scale asymmetries in the dayglow.We demonstrate the model by using the three far ultraviolet cameras on the Imager for Magnetopause-to-Aurora Global Exploration(IMAGE)mission.The method can be applied to historical missions and is relevant for upcoming missions,such as the Solar wind Magnetosphere Ionosphere Link Explorer(SMILE)mission.

Method for evaluation of geological strength index of carbonate cliff rocks:Coupled hyperspectral-digital borehole image technique

The deterioration of unstable rock mass raised interest in evaluating rock mass quality.However,the traditional evaluation method for the geological strength index(GSI)primarily emphasizes the rock structure and characteristics of discontinuities.It ignores the influence of mineral composition and shows a deficiency in assessing the integrity coefficient.In this context,hyperspectral imaging and digital panoramic borehole camera technologies are applied to analyze the mineral content and integrity of rock mass.Based on the carbonate mineral content and fissure area ratio,the strength reduction factor and integrity coefficient are calculated to improve the GSI evaluation method.According to the results of mineral classification and fissure identification,the strength reduction factor and integrity coefficient increase with the depth of rock mass.The rock mass GSI calculated by the improved method is mainly concentrated between 40 and 60,which is close to the calculation results of the traditional method.The GSI error rates obtained by the two methods are mostly less than 10%,indicating the rationality of the hyperspectral-digital borehole image coupled evaluation method.Moreover,the sensitivity of the fissure area ratio(Sr)to GSI is greater than that of the strength reduction factor(a),which means the proposed GSI is suitable for rocks with significant fissure development.The improved method reduces the influence of subjective factors and provides a reliable index for the deterioration evaluation of rock mass.

A multifunctional shear apparatus for rocks subjected to true triaxial stress and high temperature in real-time

Deep engineering disasters,such as rockbursts and collapses,are more related to the shear slip of rock joints.A novel multifunctional device was developed to study the shear failure mechanism in rocks.Using this device,the complete shearedeformation process and long-term shear creep tests could be performed on rocks under constant normal stiffness(CNS)or constant normal loading(CNL)conditions in real-time at high temperature and true-triaxial stress.During the research and development process,five key technologies were successfully broken through:(1)the ability to perform true-triaxial compressioneshear loading tests on rock samples with high stiffness;(2)a shear box with ultra-low friction throughout the entire stress space of the rock sample during loading;(3)a control system capable of maintaining high stress for a long time and responding rapidly to the brittle fracture of a rock sample as well;(4)a refined ability to measure the volumetric deformation of rock samples subjected to true triaxial shearing;and(5)a heating system capable of maintaining uniform heating of the rock sample over a long time.By developing these technologies,loading under high true triaxial stress conditions was realized.The apparatus has a maximum normal stiffness of 1000 GPa/m and a maximum operating temperature of 300C.The differences in the surface temperature of the sample are constant to within5C.Five types of true triaxial shear tests were conducted on homogeneous sandstone to verify that the apparatus has good performance and reliability.The results show that temperature,lateral stress,normal stress and time influence the shear deformation,failure mode and strength of the sandstone.The novel apparatus can be reliably used to conduct true-triaxial shear tests on rocks subjected to high temperatures and stress.

Deep learning-based inpainting of saturation artifacts in optical coherence tomography images

Limited by the dynamic range of the detector,saturation artifacts usually occur in optical coherence tomography(OCT)imaging for high scattering media.The available methods are difficult to remove saturation artifacts and restore texture completely in OCT images.We proposed a deep learning-based inpainting method of saturation artifacts in this paper.The generation mechanism of saturation artifacts was analyzed,and experimental and simulated datasets were built based on the mechanism.Enhanced super-resolution generative adversarial networks were trained by the clear–saturated phantom image pairs.The perfect reconstructed results of experimental zebrafish and thyroid OCT images proved its feasibility,strong generalization,and robustness.

SMILE soft X-ray Imager flight model CCD370 pre-flight device characterisation

Throughout the SMILE mission the satellite will be bombarded by radiation which gradually damages the focal plane devices and degrades their performance.In order to understand the changes of the CCD370s within the soft X-ray Imager,an initial characterisation of the devices has been carried out to give a baseline performance level.Three CCDs have been characterised,the two flight devices and the flight spa re.This has been carried out at the Open University in a bespo ke cleanroom measure ment facility.The results show that there is a cluster of bright pixels in the flight spa re which increases in size with tempe rature.However at the nominal ope rating tempe rature(-120℃) it is within the procure ment specifications.Overall,the devices meet the specifications when ope rating at -120℃ in 6 × 6 binned frame transfer science mode.The se rial charge transfer inefficiency degrades with temperature in full frame mode.However any charge losses are recovered when binning/frame transfer is implemented.