Low-Cost Real-Time Automated Optical Inspection Using Deep Learning and Attention Map

The recent trends in Industry 4.0 and Internet of Things have encour-aged many factory managers to improve inspection processes to achieve automa-tion and high detection rates.However,the corresponding cost results of sample tests are still used for quality control.A low-cost automated optical inspection system that can be integrated with production lines to fully inspect products with-out adjustments is introduced herein.The corresponding mechanism design enables each product to maintain afixed position and orientation during inspec-tion to accelerate the inspection process.The proposed system combines image recognition and deep learning to measure the dimensions of the thread and iden-tify its defects within 20 s,which is lower than the production-line productivity per 30 s.In addition,the system is designed to be used for monitoring production lines and equipment status.The dimensional tolerance of the proposed system reaches 0.012 mm,and a 100%accuracy is achieved in terms of the defect reso-lution.In addition,an attention-based visualization approach is utilized to verify the rationale for the use of the convolutional neural network model and identify the location of thread defects.

Improving accuracy of automatic optical inspection with machine learning

Electronic devices require the printed circuit board(PCB)to support the whole structure,but the assembly of PCBs suffers from welding problem of the electronic components such as surface mounted devices(SMDs)resistors.The automated optical inspection(AOI)machine,widely used in industrial production,can take the image of PCBs and examine the welding issue.However,the AOI machine could commit false negative errors and dedicated technicians have to be employed to pick out those misjudged PCBs.This paper proposes a machine learning based method to improve the accuracy of AOI.In particular,we propose an adjacent pixel RGB value based method to pre-process the image from the AOI machine and build a customized deep learning model to classify the image.We present a practical scheme including two machine learning procedures to mitigate AOI errors.We conduct experiments with the real dataset from a production line for three months,the experimental results show that our method can reduce the rate of misjudgment from 0.3%–0.5%to 0.02%–0.03%,which is meaningful for thousands of PCBs each containing thousands of electronic components in practice.

Optical wafer defect inspection at the 10 nm technology node and beyond

The growing demand for electronic devices, smart devices, and the Internet of Things constitutes the primary driving force for marching down the path of decreased critical dimension and increased circuit intricacy of integrated circuits. However, as sub-10 nm high-volume manufacturing is becoming the mainstream, there is greater awareness that defects introduced by original equipment manufacturer components impact yield and manufacturing costs. The identification, positioning, and classification of these defects, including random particles and systematic defects, are becoming more and more challenging at the 10 nm node and beyond.Very recently, the combination of conventional optical defect inspection with emerging techniques such as nanophotonics, optical vortices, computational imaging, quantitative phase imaging, and deep learning is giving the field a new possibility. Hence, it is extremely necessary to make a thorough review for disclosing new perspectives and exciting trends, on the foundation of former great reviews in the field of defect inspection methods. In this article, we give a comprehensive review of the emerging topics in the past decade with a focus on three specific areas:(a) the defect detectability evaluation,(b) the diverse optical inspection systems,and(c) the post-processing algorithms. We hope, this work can be of importance to both new entrants in the field and people who are seeking to use it in interdisciplinary work.

Industry-Oriented Detection Method of PCBA Defects Using Semantic Segmentation Models

Automated optical inspection(AOI)is a significant process in printed circuit board assembly(PCBA)production lines which aims to detect tiny defects in PCBAs.Existing AOI equipment has several deficiencies including low throughput,large computation cost,high latency,and poor flexibility,which limits the efficiency of online PCBA inspection.In this paper,a novel PCBA defect detection method based on a lightweight deep convolution neural network is proposed.In this method,the semantic segmentation model is combined with a rule-based defect recognition algorithm to build up a defect detection frame-work.To improve the performance of the model,extensive real PCBA images are collected from production lines as datasets.Some optimization methods have been applied in the model according to production demand and enable integration in lightweight computing devices.Experiment results show that the production line using our method realizes a throughput more than three times higher than traditional methods.Our method can be integrated into a lightweight inference system and pro-mote the flexibility of AOI.The proposed method builds up a general paradigm and excellent example for model design and optimization oriented towards industrial requirements.

Integrated Inspection of QoM, QoP, and QoS for AOI Industries in Metaverses

With the rapid development of information technologies such as digital twin, extended reality, and blockchain,the hype around "metaverse" is increasing at astronomical speed. However, much attention has been paid to its entertainment and social functions. Considering the openness and interoperability of metaverses, the market of quality inspection promises explosive growth. In this paper, taking advantage of metaverses, we first propose the concept of Automated Quality Inspection(Auto QI), which performs integrated inspection covering the entire manufacturing process, including Quality of Materials, Quality of Manufacturing(Qo M), Quality of Products, Quality of Processes(Qo P), Quality of Systems, and Quality of Services(Qo S). Based on the scenarios engineering theory, we discuss how to perform interactions between metaverses and the physical world for virtual design instruction and physical validation feedback. Then we introduce a bottomup inspection device development workflow with productivity tools offered by metaverses, making development more effective and efficient than ever. As the core of quality inspection,we propose Quality Transformers to complete detection task,while federated learning is integrated to regulate data sharing.In summary, we point out the development directions of quality inspection under metaverse tide.

Method for Detecting Weld Feature Size Based on Line Structured Light

With the rapid development of the machining and manufacturing industry,welding has been widely used in forming connections of structural parts.At present,manual methods are often used for welding and quality inspection,with low efficiency and unstable product quality.Due to the requirements of visual inspection of weld feature size,a visual inspection system for weld feature size based on line structured light(LSL)is designed and built in this paper.An adaptive light stripe sub-pixel center extraction algorithm and a feature point extraction algorithm for welding light stripe are proposed.The experiment results show that the detection error of the weld width is 0.216 mm,the detection error of the remaining height is 0.035 mm,the single measurement costs 109 ms,and the inspection stability and repeatability of the system is 1%.Our approach can meet the online detection requirements of practical applications.

Topography Measurement for Monitoring Manufacturing Processes in Harsh Conditions

High precision manufacturing, e.g. milling and grinding, which have manufacturing tolerances in the range of <10 μm require microscopic measurement techniques for the inspection of the manufactured components. These measurement techniques are very sensitive to cooling liquids and lubricants which are essential for many manufacturing processes. Therefore, the measurement of the components is usually conducted in separate and clean laboratories and not directly in the manufacturing machine. This approach has some major drawbacks, e.g. high time consumption and no possibility for online process monitoring. In this article, a novel concept for the integration of high precision optical topography measurement systems into the manufacturing machine is introduced and compared to other concepts. The introduced concept uses a reservoir with cooling liquid in which the measurement object is immersed during the measurement. Thereby, measurement disturbance by splashing cooling liquids and lubricants can effectively be avoided.

Flexible electronics manufacturing technology and equipment

Flexible electronics such as mechanically compliant displays,sensors and solar cells,have important applications in the fields of energy,national defence and biomedicine,etc.Various types of flexible electronics have been proposed or developed by the improvements in structural designs,material properties and device integrations.However,the manufacturing of flexible electronics receives little attention,which limits its mass production and industrialization.The increasing demands on the size,functionality,resolution ratio and reliability of flexible electronics bring several significant challenges in their manufacturing processes.This work aims to report the state-of-art technologies and applications of flexible electronics manufacturing.Three key technologies including electrohydrodynamic direct-writing,flip chip and automatic optical inspection are highlighted.The mechanism and developments of these technologies are discussed in detail.Based on these technologies,the present work develops three kinds of manufacturing equipment,i.e.,inkjet printing manufacturing equipment,robotized additive manufacturing equipment,and roll-to-roll manufacturing equipment.The advanced manufacturing processes,equipment and systems for flexible electronics pave the way for applications of new displays,smart sensing skins and epidermal electronics,etc.By reviewing the developments of flexible electronics manufacturing technology and equipment,it can be found that the existing advances greatly promote the applications and commercialization of flexible electronics.Since flexible electronics manufacturing contains many multi-disciplinary problems,the current investigations are confronted with great challenges.Therefore,further developments of the reviewed manufacturing technology and equipment are necessary to break the current limitations of manufacturing resolution,efficiency and reliability.

Chirp control of femtosecond-pulse scattering from drag-reducing surface-relief gratings

The role of chirp on the light-matter interaction of femto- and pico-second laser pulses with functional structured surfaces is studied using drag-reducing riblets as an example. The three-dimensional, periodic microstructure naturally gives rise to a mutual interplay of （i） reflection, （ii） scattering, and （iii） diffraction phenomena of incident coherent light. Furthermore, for femtosecond pulses, the structure induces （iv） an optical delay equivalent to a consecutive temporal delay of 230 fs in places of the pulse. These features enable studying experimentally and numerically the effect of tuning both pulse duration τ and spectral bandwidth Δω on the features of the wideangle scattering pattern from the riblet structure. As a result, we discovered a significant breakdown of fringes in the scattering pattern with decreasing pulse duration and/or increasing spectral bandwidth. This unique type of chirp control is straightforward/y explained and verified by numerical modeling considering the spectral and temporal interaction between different segments within the scattered, linearly chirped pulse and the particular geometric features of the riblet structure. The visibility of the fringe pattern can be precisely adjusted, and the offstate is achieved using τ 〈 230 fs or Δω〉 2.85 × 10^13 rad/s.

Optic flaws detection and location based on a plenoptic camera

In this Letter,we propose an on-line inspection method based on a plenoptic camera to detect and locate flaws of optics.Specifically,due to the extended depth of field of the plenoptic camera,a series of optics can be inspected efficiently and simultaneously.Moreover,the depth estimation capability of the plenoptic camera allows for locating flaws while detecting them.Besides,the detection and location can be implemented with a single snapshot of the plenoptic camera.Consequently,this method provides us with the opportunity to reduce the cost of time and labor of inspection and remove the flaw optics,which may lead to performance degradation of optical systems.