A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection

Background Assembly guided by paper documents is the most widespread type used in the process of aircraft cable assembly.This process is very complicated and requires assembly workers with high-level skills.The technologies of wearable Augmented Reality(AR)and portable visual inspection can be exploited to improve the efficiency and the quality of cable assembly.Methods In this study,we propose a smart assistance system for cable assembly that combines wearable AR with portable visual inspection.Specifically,a portable visual device based on binocular vision and deep learning is developed to realize fast detection and recognition of cable brackets that are installed on aircraft airframes.A Convolutional Neural Network(CNN)is then developed to read the texts on cables after images are acquired from the camera of the wearable AR device.An authoring tool that was developed to create and manage the assembly process is proposed to realize visual guidance of the cable assembly process based on a wearable AR device.The system is applied to cable assembly on an aircraft bulkhead prototype.Results The results show that this system can recognize the number,types,and locations of brackets,and can correctly read the text of aircraft cables.The authoring tool can assist users who lack professional programming experience in establishing a process plan,i.e.,assembly outline based on AR for cable assembly.Conclusions The system can provide quick assembly guidance for aircraft cable with texts,images,and a 3 D model.It is beneficial for reducing the dependency on paper documents,labor intensity,and the error rate.

Development of an automatic weld surface appearance inspection system using machine vision

In this paper, an automatic inspection system for weld surface appearance using machine vision has been developed to recognize weld surface defects such as porosities, cracks, etc. It can replace conventional manual visual inspection method, which is tedious, time-consuming, subjective, experience-depended, and sometimes biased. The system consists of a CCD camera, a self-designed annular light source, a sensor controller, a frame grabbing card, a computer and so on. After acquiring weld surface appearance images using CCD, the images are preprocessed using median filtering and a series of image enhancement algorithms. Then a dynamic threshold and morphology algorithms are applied to segment defect object. Finally, defect features information is obtained by eight neighborhoods boundary chain code algorithm. Experimental results show that the developed system is capable of inspecting most surface defects such as porosities, cracks with high reliability and accuracy.

Defect inspection of indoor components in buildings using deep learning object detection and augmented reality

Visual inspection is commonly adopted for building operation,maintenance,and safety.The durability and defects of components or materials in buildings can be quickly assessed through visual inspection.However,implementations of visual inspection are substantially time-consuming,labor-intensive,and error-prone because useful auxiliary tools that can instantly highlight defects or damage locations from images are not available.Therefore,an advanced building inspection framework is developed and implemented with augmented reality(AR)and real-time damage detection in this study.In this framework,engineers should walk around and film every corner of the building interior to generate the three-dimensional(3D)environment through ARKit.Meanwhile,a trained YOLOv5 model real-time detects defects during this process,even in a large-scale field,and the defect locations indicating the detected defects are then marked in this 3D environment.The defects areas can be measured with centimeter-level accuracy with the light detection and ranging(LiDAR)on devices.All required damage information,including defect positions and sizes,is collected at a time and can be rendered in the 2D and 3D views.Finally,this visual inspection can be efficiently conducted,and the previously generated environment can also be loaded to re-localize existing defect marks for future maintenance and change observation.Moreover,the proposed framework is also implemented and verified by an underground parking lot in a building to detect and quantify surface defects on concrete components.As seen in the results,the conventional building inspection is significantly improved with the aid of the proposed framework in terms of damage localization,damage quantification,and inspection efficiency.

SpyGlass application for duodenoscope working channel inspection: Impact on the microbiological surveillance

BACKGROUND Patient-ready duodenoscopes were designed with an assumed contamination rate of less than 0.4%;however,it has been reported that 5.4%of clinically used duodenoscopes remain contaminated with viable high-concern organisms despite following the manufacturer’s instructions.Visual inspection of working channels has been proposed as a quality control measure for endoscope reprocessing.There are few studies related to this issue.AIM To investigate the types,severity rate,and locations of abnormal visual inspection findings inside patient-ready duodenoscopes and their microbiological significance.METHODS Visual inspections of channels were performed in 19 patient-ready duodenoscopes using the SpyGlass visualization system in two endoscopy units of tertiary care teaching hospitals(Tri-Service General Hospital and National Taiwan University Hospital)in Taiwan.Inspections were recorded and reviewed to evaluate the presence of channel scratches,buckling,stains,debris,and fluids.These findings were used to analyze the relevance of microbiological surveillance.RESULTS Seventy-two abnormal visual inspection findings in the 19 duodenoscopes were found,including scratches(n=10,52.6%),buckling(n=15,78.9%),stains(n=14,73.7%),debris(n=14,73.7%),and fluids(n=6,31.6%).Duodenoscopes>12 mo old had a significantly higher number of abnormal visual inspection findings than those≤12 mo old(46 findings vs 26 findings,P<0.001).Multivariable regression analyses demonstrated that the bending section had a significantly higher risk of being scratched,buckled,and stained,and accumulating debris than the insertion tube.Debris and fluids showed a significant positive correlation with microbiological contamination(P<0.05).There was no significant positive Spearman’s correlation coefficient between negative bacterial cultures and debris,between that and fluids,and the concomitance of debris and fluids.This result demonstrated that the presence of fluid and debris was associated with positive cultures,but not negative cultures.Further multivariate analysis demonstrated that fluids,but not debris,is an independent factor for bacterial culture positivity.CONCLUSION In patient-ready duodenoscopes,scratches,buckling,stains,debris,and fluids inside the working channel are common,which increase the microbiological contamination susceptibility.The SpyGlass visualization system may be recommended to identify suboptimal reprocessing.

Assembly Auxiliary System for Narrow Cabins of Spacecraft

Due to the narrow space and complex structure of spacecraft cabin, the existing asssembly systems can not well suit for the assembly process of cabin products. This paper aims to introduce an assembly auxiliary system for cabin products. A hierarchical-classification method is proposed to re-adjust the initial assembly relationship of cabin into a new hierarchical structure for efficient assembly planning. An improved ant colony algorithm based on three assembly principles is established for searching a optimizational assembly sequence of cabin parts. A mixed reality assembly environment is constructed with enhanced inforamtion to promote interaction efficiency of assembly training and guidance. Based on the machine vision technology, the inspection of left redundant objects and measurement of parts distance in inner cabin are efficiently performed. The proposed system has been applied to the assembly work of a spacecraft cabin with 107 parts, which includes cabin assembly planning, assembly training and assembly quality inspection. The application result indicates that the proposed system can be an effective assistant tool to cabin assembly works and provide an intuitive and real assembly experience for workers. This paper presents an assembly auxiliary system for spacecraft cabin products, which can provide technical support to the spacecraft cabin assembly industry.

Posture Adjustment of Microphone Based on Image Recognition in Automatic Welding System

As the requirements of production process is getting higher and higher with the reduction of volume,microphone production automation become an urgent need to improve the production efficiency.The most important part is studied and a precise algorithm of calculating the deviation angle of four types microphones is proposed,based on the feature extraction and visual detection.Pretreatment is performed to achieve the real-time microphone image.Canny edge detection and typical feature extraction are used to distinguish the four types of microphones,categorizing them as type M1 and type M2.And Hough transformation is used to extract the image features of microphone.Therefore,the deviation angle between the posture of microphone and the ideal posture in 2Dplane can be achieved.Depending on the angle,the system drives the motor to adjust posture of the microphone.The final purpose is to realize the high efficiency welding of four different types of microphones.

An Overview of Seismic Risk Management for Italian Architectural Heritage

The frequent occurrence of seismic events in Italy poses a strategic problem that involves either the culture of preservation of historical heritage or the civil protection action aimed to reduce the risk to people and goods(buildings,bridges,dams,slopes,etc.).Most of the Italian architectural heritage is vulnerable to earthquakes,identifying the vulnerability as the inherent predisposition of the masonry building to suffer damage and collapse during an earthquake.In fact,the structural concept prevailing in these ancient masonry buildings is aimed at ensuring prevalent resistance to vertical gravity loads.Rarely do these ancient masonry structures offer relevant resistance to actions other than vertical ones and then they are intrinsically vulnerable to stresses induced by the earthquakes.One of the main technical activities carried out by the Civil Protection after an earthquake is to assess the damage in the buildings and to evaluate their own usability.Regarding historical buildings,ad-hoc synthetic forms,drafted in agreement between the Italian Civil Protection Department and the Ministry of Cultural Heritage and Activities and Tourism and based on visual inspection,are adopted by qualified technicians.In this paper,such activities are described and discussed along with the Italian Civil Protection System.However,given the complexity of the main technical activities to be performed after an earthquake there is a need for more accurate methods based on Structural Health Monitoring.

Screening for Precancerous Cervical Lesions in Women of Reproductive Age in the Kara Region of Togo in 2022

Introduction: Worldwide, cervical cancer is the 4th most common cancer in women and is a public health problem. The objective of this study was to estimate the prevalence of precancerous cervical lesions and to describe its associated factors among women of reproductive age in the Kara region of Togo. Methods: A cross-sectional study was conducted from March 6 to 14, 2022 in 11 health centers in Kara. Data were collected using a standardized questionnaire and screening for precancerous cervical lesions was performed by visual inspection of the cervix, after application of 5% acetic acid and Lugol’s. Logistic regression analysis was performed to describe factors associated with precancerous lesions. Results: A total of 728 women with a median age of 36, interquartile range [31 - 41] were enrolled. The prevalence of precancerous cervical lesions was 3.9%, 95% confidence interval (95% CI: [2.6 - 5.4]. Factors associated with the presence of precancerous lesions were age at first sexual intercourse ≥ 18 years (adjusted odds ratio (aOR = 3.67;95% CI [1.17 - 18.4]) and being a sex worker (aOR = 8.14;95% CI [1.96 - 27.1]). HIV infection was not associated with the presence of precancerous lesions. Conclusion: The results of this study underscore the importance of intensifying cervical cancer screening efforts in resource-limited countries for better management. These efforts should prioritize vulnerable populations such as female sex workers.

Discussion on Maintenance Cases of Wind Turbine Components

At present,the greenhouse effect is caused by excessive emission of carbon dioxide.As a result the Arctic ice has melted and sea levels have risen.If it continues to deteriorate,it will cause human catastrophe.In order to avoid direct crisis and development,green energy is the only necessary way.Here,wind power plays an important role.Onshore wind power has been developed in Taiwan for more than 15 years.There are 341 onshore wind turbines that have been built so far.The total installed capacity is 678 MW high.Among them,Tai power occupies a total of 169 stations with a total installed capacity of 294 MW.Offshore wind turbines are also under construction.By 2025,the capacity will be 5 to 6 GW.It can be seen that the supply of wind power in the overall power market will become an important area in the future.Therefore,how to improve the availability and capacity factors of wind turbine power generation will become a top priority for owners.Since most of the world’s best wind farms are in the Taiwan Strait,this is a unique feature of Taiwan,although Taiwan lacks traditional fuels,petroleum,coal,natural gas and other resources.If these abundant solar and wind energy resources can be effectively utilized,in addition to reducing carbon emissions and contributing to the world,the development of green energy can also drive the development of the domestic green energy industry,also through the development of green energy to establish domestic operation and maintenance technology for wind turbines.

A review of applications of visual inspection technology based on image processing in the railway industry

In order to ensure the safety of railway transportation,it is necessary to regularly check for faults and defects in the railway system.Visual inspection technology is conducive to improving the low efficiency,poor economy and inaccurate detection results of traditional detection methods.This paper introduces the research and contribution of various scholars in the field of visual inspection,summarizes the application and development of visual inspection technology in the railway industry,and finally forecasts the future research direction of visual inspection technology.

Imagery Enhancement and Interpretation for Remote Visual Inspection of Aging Civil Infrastructure

In this paper, we describe an image enhancement and interpretation methodology to enhance and recognize surface defects and critical patterns from remote imagery of sewer pipeline inspection. The objective is to provide inspectors and professionals with better tools to allow them to examine the imagery for condition assessment. We present initial results of a collaboration with a robotic company through a case study on computer-assisted processing and interpretation of sewer pipeline inspection imagery. In the mean time, the described enhancement and interpretation methodology can also be applied to sewer pipeline condition assessment in an offline mode, where this methodology can support professionals’ examination of acquired sewer condition imagery.

Methodology for Road Defect Detection and Administration Based on Mobile Mapping Data

A detailed inspection of roads requires highly detailed spatial data with sufficient precision to deliver an accurate geometry and to describe road defects visually.This paper presents a novel method for the detection of road defects.The input data for road defect detection included point clouds and orthomosaics gathered by mobile mapping technology.The defects were categorized in three major groups with the following geometric primitives:points,lines and polygons.The method suggests the detection of point objects from matched point clouds,panoramic images and ortho photos.Defects were mapped as point,line or polygon geometries,directly derived from orthomosaics and panoramic images.Besides the geometric position of road defects,all objects were assigned to a variety of attributes:defect type,surface material,center-of-gravity,area,length,corresponding image of the defect and degree of damage.A spatial dataset comprising defect values with a matching data type was created to perform the attribute analysis quickly and correctly.The final product is a spatial vector data set,consisting of points,lines and polygons,which contains attributes with further information and geometry.This paper demonstrates that mobile mapping suits a large-scale feature extraction of road infrastructure defects.By its simplicity and flexibility,the presented methodology allows it to be easily adapted to extract further feature types with their attributes.This makes the proposed approach a vital tool for data extraction settings with multiple mobile mapping data analysts,e.g.,offline crowdsourcing.

Build Gaussian Distribution Under Deep Features for Anomaly Detection and Localization

Anomaly detection in images has attracted a lot of attention in the field of computer vision.It aims at identifying images that deviate from the norm and segmenting the defect within images.However,anomalous samples are difficult to collect comprehensively,and labeled data is costly to obtain in many practical scenarios.We proposes a simple framework for unsupervised anomaly detection.Specifically,the proposed method directly employs CNN pre-trained on ImageNet to extract deep features from normal images and reduce dimensionality based on Principal Components Analysis(PCA),then build the distribution of normal features via the multivariate Gaussian(MVG),and determine whether the test image is an abnormal image according to Mahalanobis distance.We further investigate which features are most effective in detecting anomalies.Extensive experiments on the MVTec anomaly detection dataset show that the proposed method achieves 98.6%AUROC in image-level anomaly detection and outperforms previous methods by a large margin.

一种感官检验用玉米不完善粒标准样品的制备

Maize unsound kernel content is one of the limited items in maize trade, and generally determined by sensory detection. A certified reference material(CRM) for visual inspection of maize unsound kernels was developed according to a national standard of China, GB 1353-2018, and five items, such as insect-damaged kernels, spotted kernels, broken kernels, sprouted kernels and moldy kernels, were included. Unsound maize kernels were collected from fields or prepared in a laboratory, then screened, and ten kernels demonstrating varying levels of unsoundness or damage for each item were embedded in epoxy resin. The CRM showed excellent homogeneity and stability, which was stable for 14 days at the temperature from-20℃ to 45℃ and for at least 18 months at room temperature out of direct sunlight and strong light. Co-laboratory confirmation showed the CRM conformed to the morphological characteristics described in GB 1353-2018. The research filled the gap of unsound maize kernel CRM in maize detection.

Automated classification of civil structure defects based on convolutional neural network

Today,the most commonly used civil infrastructure inspection method is based on a visual assessment conducted by certified inspectors following prescribed protocols.However,the increase in aggressive environmental and load conditions,coupled with the achievement of many structures of the life-cycle end,has highlighted the need to automate damage identification and satisfy the number of structures that need to be inspected.To overcome this challenge,this paper presents a method for automating concrete damage classification using a deep convolutional neural network.The convolutional neural network was designed after an experimental investigation of a wide number of pretrained networks,applying the transfer-learning technique.Training and validation were conducted using a database built with 1352 images balanced between“undamaged”,“cracked”,and“delaminated”concrete surfaces.To increase the network robustness compared to images in real-world situations,different image configurations have been collected from the Internet and on-field bridge inspections.The GoogLeNet model,with the highest validation accuracy of approximately 94%,was selected as the most suitable network for concrete damage classification.The results confirm that the proposed model can correctly classify images from real concrete surfaces of bridges,tunnels,and pavement,resulting in an effective alternative to the current visual inspection techniques.

A mask R-CNN based method for inspecting cable brackets in aircraft

In the aviation industry,cable bracket is one of the most common parts.The traditional assembly state inspection method of cable bracket is to manually compare by viewing 3 D models.The purpose of this paper is to address the problem of inefficiency of traditional inspection method.In order to solve the problem that machine learning algorithm requires large dataset and manually labeling of dataset is a laborious and time-consuming task,a simulation platform is developed to automatically generate synthetic realistic brackets images with pixel-level annotations based on 3 D digital mock-up.In order to obtain accurate shapes of brackets from 2 D image,a brackets recognizer based on Mask R-CNN is trained.In addition,a semi-automatic cable bracket inspection method is proposed.With this method,the inspector can easily obtain the inspection result only by taking a picture with a portable device,such as augmented reality(AR)glasses.The inspection task will be automatically executed via bracket recognition and matching.The experimental result shows that the proposed method for automatically labeling dataset is valid and the proposed cable bracket inspection method can effectively inspect cable bracket in the aircraft.Finally,a prototype system based on client-server framework has been developed for validation purpose.