An FPGA-based real-time image processing system

This paper analyzes the current difficulties encountered in on-line inspection systems of strip surface quality, specifically relating to problems with real-time processing of huge amounts of data. To address this need, this paper describes an FPGA-based high-speed image processing module with both hardware and software aspects. Improving these two aspects together will help the system achieve real-time processing of massive image data, and simplifies the architecture of the strip surface quality on-line inspection system.

A Versatile High-speed Image Processing System Based on DSP and CPLD

In this paper, we present an optimized design method for high-speed embedded image processing system using 32 bit floating-point Digital Signal Processor （DSP） and Complex Programmable Logic Device （CPLD）. The DSP acts as the main processor of the system： executes digital image processing algorithms and operates other devices such as image sensor and CPLD. The CPLD is used to acquire images and achieve complex logic control of the whole system. Some key technologies are introduced to enhance the performance of our system. In particular, the use of DSP/BIOS tool to develop DSP applications makes our program run much more efficiently. As a result, this system can provide an excellent computing platform not only for executing complex image processing algorithms, but also for other digital signal processing or multi-channel data collection by choosing different sensors or Analog-to-Digital （A/D） converters.

Parallel Technologies with Image Processing Using Inverse Filter

Real-time capabilities and computational efficiency are provided by parallel image processing utilizing OpenMP. However, race conditions can affect the accuracy and reliability of the outcomes. This paper highlights the importance of addressing race conditions in parallel image processing, specifically focusing on color inverse filtering using OpenMP. We considered three solutions to solve race conditions, each with distinct characteristics: #pragma omp atomic: Protects individual memory operations for fine-grained control. #pragma omp critical: Protects entire code blocks for exclusive access. #pragma omp parallel sections reduction: Employs a reduction clause for safe aggregation of values across threads. Our findings show that the produced images were unaffected by race condition. However, it becomes evident that solving the race conditions in the code makes it significantly faster, especially when it is executed on multiple cores.

Parallel Image Processing: Taking Grayscale Conversion Using OpenMP as an Example

In recent years, the widespread adoption of parallel computing, especially in multi-core processors and high-performance computing environments, ushered in a new era of efficiency and speed. This trend was particularly noteworthy in the field of image processing, which witnessed significant advancements. This parallel computing project explored the field of parallel image processing, with a focus on the grayscale conversion of colorful images. Our approach involved integrating OpenMP into our framework for parallelization to execute a critical image processing task: grayscale conversion. By using OpenMP, we strategically enhanced the overall performance of the conversion process by distributing the workload across multiple threads. The primary objectives of our project revolved around optimizing computation time and improving overall efficiency, particularly in the task of grayscale conversion of colorful images. Utilizing OpenMP for concurrent processing across multiple cores significantly reduced execution times through the effective distribution of tasks among these cores. The speedup values for various image sizes highlighted the efficacy of parallel processing, especially for large images. However, a detailed examination revealed a potential decline in parallelization efficiency with an increasing number of cores. This underscored the importance of a carefully optimized parallelization strategy, considering factors like load balancing and minimizing communication overhead. Despite challenges, the overall scalability and efficiency achieved with parallel image processing underscored OpenMP’s effectiveness in accelerating image manipulation tasks.

Real Time Thermal Image Based Machine Learning Approach for Early Collision Avoidance System of Snowplows

In an effort to reduce vehicle collisions with snowplows in poor weather conditions, this paper details the development of a real time thermal image based machine learning approach to an early collision avoidance system for snowplows, which intends to detect and estimate the distance of trailing vehicles. Due to the operational conditions of snowplows, which include heavy-blowing snow, traditional optical sensors like LiDAR and visible spectrum cameras have reduced effectiveness in detecting objects in such environments. Thus, we propose using a thermal infrared camera as the primary sensor along with machine learning algorithms. First, we curate a large dataset of thermal images of vehicles in heavy snow conditions. Using the curated dataset, two machine-learning models based on the modified ResNet architectures were trained to detect and estimate the trailing vehicle distance using real-time thermal images. The trained detection network was capable of detecting trailing vehicles 99.0% of the time at 1500.0 ft distance from the snowplow. The trained trailing distance network was capable of estimating distance with an average estimation error of 10.70 ft. The inference performance of the trained models is discussed, along with the interpretation of the performance.

Automated deep learning system for power line inspection image analysis and processing: architecture and design issues

The continuous growth in the scale of unmanned aerial vehicle (UAV) applications in transmission line inspection has resulted in a corresponding increase in the demand for UAV inspection image processing. Owing to its excellent performance in computer vision, deep learning has been applied to UAV inspection image processing tasks such as power line identification and insulator defect detection. Despite their excellent performance, electric power UAV inspection image processing models based on deep learning face several problems such as a small application scope, the need for constant retraining and optimization, and high R&D monetary and time costs due to the black-box and scene data-driven characteristics of deep learning. In this study, an automated deep learning system for electric power UAV inspection image analysis and processing is proposed as a solution to the aforementioned problems. This system design is based on the three critical design principles of generalizability, extensibility, and automation. Pre-trained models, fine-tuning (downstream task adaptation), and automated machine learning, which are closely related to these design principles, are reviewed. In addition, an automated deep learning system architecture for electric power UAV inspection image analysis and processing is presented. A prototype system was constructed and experiments were conducted on the two electric power UAV inspection image analysis and processing tasks of insulator self-detonation and bird nest recognition. The models constructed using the prototype system achieved 91.36% and 86.13% mAP for insulator self-detonation and bird nest recognition, respectively. This demonstrates that the system design concept is reasonable and the system architecture feasible .

Correlations between mineral composition and mechanical properties of granite using digital image processing and discrete element method

This study investigated the correlations between mechanical properties and mineralogy of granite using the digital image processing(DIP) and discrete element method(DEM). The results showed that the X-ray diffraction(XRD)-based DIP method effectively analyzed the mineral composition contents and spatial distributions of granite. During the particle flow code(PFC2D) model calibration phase, the numerical simulation exhibited that the uniaxial compressive strength(UCS) value, elastic modulus(E), and failure pattern of the granite specimen in the UCS test were comparable to the experiment. By establishing 351 sets of numerical models and exploring the impacts of mineral composition on the mechanical properties of granite, it indicated that there was no negative correlation between quartz and feldspar for UCS, tensile strength(σ_(t)), and E. In contrast, mica had a significant negative correlation for UCS, σ_(t), and E. The presence of quartz increased the brittleness of granite, whereas the presence of mica and feldspar increased its ductility in UCS and direct tensile strength(DTS) tests. Varying contents of major mineral compositions in granite showed minor influence on the number of cracks in both UCS and DTS tests.

Analysis of morphological characteristics of gravels based on digital image processing technology and self-organizing map

A comprehensive understanding of spatial distribution and clustering patterns of gravels is of great significance for ecological restoration and monitoring.However,traditional methods for studying gravels are low-efficiency and have many errors.This study researched the spatial distribution and cluster characteristics of gravels based on digital image processing technology combined with a self-organizing map(SOM)and multivariate statistical methods in the grassland of northern Tibetan Plateau.Moreover,the correlation of morphological parameters of gravels between different cluster groups and the environmental factors affecting gravel distribution were analyzed.The results showed that the morphological characteristics of gravels in northern region(cluster C)and southern region(cluster B)of the Tibetan Plateau were similar,with a low gravel coverage,small gravel diameter,and elongated shape.These regions were mainly distributed in high mountainous areas with large topographic relief.The central region(cluster A)has high coverage of gravels with a larger diameter,mainly distributed in high-altitude plains with smaller undulation.Principal component analysis(PCA)results showed that the gravel distribution of cluster A may be mainly affected by vegetation,while those in clusters B and C could be mainly affected by topography,climate,and soil.The study confirmed that the combination of digital image processing technology and SOM could effectively analyzed the spatial distribution characteristics of gravels,providing a new mode for gravel research.

Damage detection with image processing: a comparative study

Large structures,such as bridges,highways,etc.,need to be inspected to evaluate their actual physical and functional condition,to predict future conditions,and to help decision makers allocating maintenance and rehabilitation resources.The assessment of civil infrastructure condition is carried out through information obtained by inspection and/or monitoring operations.Traditional techniques in structural health monitoring(SHM)involve visual inspection related to inspection standards that can be time-consuming data collection,expensive,labor intensive,and dangerous.To address these limitations,machine vision-based inspection procedures have increasingly been investigated within the research community.In this context,this paper proposes and compares four different computer vision procedures to identify damage by image processing:Otsu method thresholding,Markov random fields segmentation,RGB color detection technique,and K-means clustering algorithm.The first method is based on segmentation by thresholding that returns a binary image from a grayscale image.The Markov random fields technique uses a probabilistic approach to assign labels to model the spatial dependencies in image pixels.The RGB technique uses color detection to evaluate the defect extensions.Finally,K-means algorithm is based on Euclidean distance for clustering of the images.The benefits and limitations of each technique are discussed,and the challenges of using the techniques are highlighted.To show the effectiveness of the described techniques in damage detection of civil infrastructures,a case study is presented.Results show that various types of corrosion and cracks can be detected by image processing techniques making the proposed techniques a suitable tool for the prediction of the damage evolution in civil infrastructures.

Automatic recognition of defects in plasma-facing material using image processing technology

Observing and analyzing surface images is critical for studying the interaction between plasma and irradiated plasma-facing materials.This paper presents a method for the automatic recognition of bubbles in transmission electron microscope(TEM)images of W nanofibers using image processing techniques and convolutional neural network(CNN).We employ a three-stage approach consisting of Otsu,local-threshold,and watershed segmentation to extract bubbles from noisy images.To address over-segmentation,we propose a combination of area factor and radial pixel intensity scanning.A CNN is used to recognize bubbles,outperforming traditional neural network models such as Alex Net and Google Net with an accuracy of 97.1%and recall of 98.6%.Our method is tested on both clear and blurred TEM images,and demonstrates humanlike performance in recognizing bubbles.This work contributes to the development of quantitative image analysis in the field of plasma-material interactions,offering a scalable solution for analyzing material defects.Overall,this study's findings establish the potential for automatic defect recognition and its applications in the assessment of plasma-material interactions.This method can be employed in a variety of specialties,including plasma physics and materials science.

Image processing based three-dimensional model reconstruction for cross-platform numerical simulation

Numerical simulation is the most powerful computational and analysis tool for a large variety of engineering and physical problems.For a complex problem relating to multi-field,multi-process and multi-scale,different computing tools have to be developed so as to solve particular fields at different scales and for different processes.Therefore,the integration of different types of software is inevitable.However,it is difficult to perform the transfer of the meshes and simulated results among software packages because of the lack of shared data formats or encrypted data formats.An image processing based method for three-dimensional model reconstruction for numerical simulation was proposed,which presents a solution to the integration problem by a series of slice or projection images obtained by the post-processing modules of the numerical simulation software.By means of mapping image pixels to meshes of either finite difference or finite element models,the geometry contour can be extracted to export the stereolithography model.The values of results,represented by color,can be deduced and assigned to the meshes.All the models with data can be directly or indirectly integrated into other software as a continued or new numerical simulation.The three-dimensional reconstruction method has been validated in numerical simulation of castings and case studies were provided in this study.

A Dual Model Watermarking Framework for Copyright Protection in Image Processing Networks

Image processing networks have gained great success in many fields,and thus the issue of copyright protection for image processing networks hasbecome a focus of attention. Model watermarking techniques are widely usedin model copyright protection, but there are two challenges: (1) designinguniversal trigger sample watermarking for different network models is stilla challenge;(2) existing methods of copyright protection based on trigger swatermarking are difficult to resist forgery attacks. In this work, we propose adual model watermarking framework for copyright protection in image processingnetworks. The trigger sample watermark is embedded in the trainingprocess of the model, which can effectively verify the model copyright. And wedesign a common method for generating trigger sample watermarks based ongenerative adversarial networks, adaptively generating trigger sample watermarksaccording to different models. The spatial watermark is embedded intothe model output. When an attacker steals model copyright using a forgedtrigger sample watermark, which can be correctly extracted to distinguishbetween the piratical and the protected model. The experiments show that theproposed framework has good performance in different image segmentationnetworks of UNET, UNET++, and FCN (fully convolutional network), andeffectively resists forgery attacks.

Surface Inspection System for Cold Rolled Strips Based on Image ProcessingTechnique

A new surface inspection system for cold rolled strips based on image processing is introduced. The system is equipped withtwo different illumination structures and CCD matrix cameras. The structure and image processing of the inspection system are described. Some efficient algorithms for image processing and classification are presented. The system is tested with strip samples fromcold rolling plants. The results show that the system can detect and recognize six common defects of cold rolled strips successfully.

HyperDB：A Hyperspectral Land Class Database Designed for an Image Processing System

Hyperspectral remote sensing is becoming more and more important amongst remote sensing techniques. In this paper, we present a hyperspectral database（Hyper DB） designed to cooperate with an embedded hyperspectral image processing system developed by the authors. Hyperspectral data are recognized and categorized by their land coverage class and band information, and can be imported from various sources such as airborne and spaceborne sensors carried by airplanes or satellites, as well as handhold instruments based on in situ ground observations. Spectral library files can be easily stored, indexed, viewed, and exported. Since Hyper DB follows standard design principles—independence, data safety, and compatibility—it satisfies the practical demand for managing categorized hyperspectral data, and can be readily expanded to other peripheral applications.

Study on the image processing of laser vision seam tracking system

Seam image processing is the basis of the realization of automatic laser vision seam tracking system, and it has become one of the important research directions. Adding windows processing, gray processing, fast median filtering, binary processing and image edge extraction are used to pretreat the seam image. In the post-processing of seam image, the feature points of the target image are succesfully detected by using center line extraction and feature points detection algorithm based on slope analysis. The whole processing time is less than 150 ms, and the real-time processing of seam image can be implemented.

Sectional Dimensions Identification of Metal Profile by Image Processing

In steel plants, estimation of the production system characteristic is highly critical to adjust the system parameters for best efficiency. Although the system parameters may be tuned very well, due to the machine and human factors involved in the production line some deficiencies may occur in product. It is important to detect such problems as early as possible. Surface defects and dimensional deviations are the most important quality problems. In this study, it is aimed to develop an approach to measure the dimensions of metal profiles by obtaining images of them. This will be of use in detecting the deviations in dimensions. A platform was introduced to simulate the real-time environment and images were taken from the metal profile using 4 laser light sources. The shape of the material is generated by combining the images taken from different cameras. Real dimensions were obtained by using image processing and mathematical conversion operations on the images. The results obtained with small deviations from the real values showed that this method can be applied in a real-time production line.

Scale-space effect and scale hybridization in image intelligent recognition of geological discontinuities on rock slopes

Geological discontinuity(GD)plays a pivotal role in determining the catastrophic mechanical failure of jointed rock masses.Accurate and efficient acquisition of GD networks is essential for characterizing and understanding the progressive damage mechanisms of slopes based on monitoring image data.Inspired by recent advances in computer vision,deep learning(DL)models have been widely utilized for image-based fracture identification.The multi-scale characteristics,image resolution and annotation quality of images will cause a scale-space effect(SSE)that makes features indistinguishable from noise,directly affecting the accuracy.However,this effect has not received adequate attention.Herein,we try to address this gap by collecting slope images at various proportional scales and constructing multi-scale datasets using image processing techniques.Next,we quantify the intensity of feature signals using metrics such as peak signal-to-noise ratio(PSNR)and structural similarity(SSIM).Combining these metrics with the scale-space theory,we investigate the influence of the SSE on the differentiation of multi-scale features and the accuracy of recognition.It is found that augmenting the image's detail capacity does not always yield benefits for vision-based recognition models.In light of these observations,we propose a scale hybridization approach based on the diffusion mechanism of scale-space representation.The results show that scale hybridization strengthens the tolerance of multi-scale feature recognition under complex environmental noise interference and significantly enhances the recognition accuracy of GD.It also facilitates the objective understanding,description and analysis of the rock behavior and stability of slopes from the perspective of image data.

Automatic area estimation of algal blooms in water bodies from UAV images using texture analysis

Algal blooms,the spread of algae on the surface of water bodies,have adverse effects not only on aquatic ecosystems but also on human life.The adverse effects of harmful algal blooms(HABs)necessitate a convenient solution for detection and monitoring.Unmanned aerial vehicles(UAVs)have recently emerged as a tool for algal bloom detection,efficiently providing on-demand images at high spatiotemporal resolutions.This study developed an image processing method for algal bloom area estimation from the aerial images(obtained from the internet)captured using UAVs.As a remote sensing method of HAB detection,analysis,and monitoring,a combination of histogram and texture analyses was used to efficiently estimate the area of HABs.Statistical features like entropy(using the Kullback-Leibler method)were emphasized with the aid of a gray-level co-occurrence matrix.The results showed that the orthogonal images demonstrated fewer errors,and the morphological filter best detected algal blooms in real time,with a precision of 80%.This study provided efficient image processing approaches using on-board UAVs for HAB monitoring.

Detection of Oscillations in Process Control Loops From Visual Image Space Using Deep Convolutional Networks

Oscillation detection has been a hot research topic in industries due to the high incidence of oscillation loops and their negative impact on plant profitability.Although numerous automatic detection techniques have been proposed,most of them can only address part of the practical difficulties.An oscillation is heuristically defined as a visually apparent periodic variation.However,manual visual inspection is labor-intensive and prone to missed detection.Convolutional neural networks(CNNs),inspired by animal visual systems,have been raised with powerful feature extraction capabilities.In this work,an exploration of the typical CNN models for visual oscillation detection is performed.Specifically,we tested MobileNet-V1,ShuffleNet-V2,Efficient Net-B0,and GhostNet models,and found that such a visual framework is well-suited for oscillation detection.The feasibility and validity of this framework are verified utilizing extensive numerical and industrial cases.Compared with state-of-theart oscillation detectors,the suggested framework is more straightforward and more robust to noise and mean-nonstationarity.In addition,this framework generalizes well and is capable of handling features that are not present in the training data,such as multiple oscillations and outliers.

Integer multiple quantum image scaling based on NEQR and bicubic interpolation

As a branch of quantum image processing,quantum image scaling has been widely studied.However,most of the existing quantum image scaling algorithms are based on nearest-neighbor interpolation and bilinear interpolation,the quantum version of bicubic interpolation has not yet been studied.In this work,we present the first quantum image scaling scheme for bicubic interpolation based on the novel enhanced quantum representation(NEQR).Our scheme can realize synchronous enlargement and reduction of the image with the size of 2^(n)×2^(n) by integral multiple.Firstly,the image is represented by NEQR and the original image coordinates are obtained through multiple CNOT modules.Then,16 neighborhood pixels are obtained by quantum operation circuits,and the corresponding weights of these pixels are calculated by quantum arithmetic modules.Finally,a quantum matrix operation,instead of a classical convolution operation,is used to realize the sum of convolution of these pixels.Through simulation experiments and complexity analysis,we demonstrate that our scheme achieves exponential speedup over the classical bicubic interpolation algorithm,and has better effect than the quantum version of bilinear interpolation.