A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection

Background Assembly guided by paper documents is the most widespread type used in the process of aircraft cable assembly.This process is very complicated and requires assembly workers with high-level skills.The technologies of wearable Augmented Reality(AR)and portable visual inspection can be exploited to improve the efficiency and the quality of cable assembly.Methods In this study,we propose a smart assistance system for cable assembly that combines wearable AR with portable visual inspection.Specifically,a portable visual device based on binocular vision and deep learning is developed to realize fast detection and recognition of cable brackets that are installed on aircraft airframes.A Convolutional Neural Network(CNN)is then developed to read the texts on cables after images are acquired from the camera of the wearable AR device.An authoring tool that was developed to create and manage the assembly process is proposed to realize visual guidance of the cable assembly process based on a wearable AR device.The system is applied to cable assembly on an aircraft bulkhead prototype.Results The results show that this system can recognize the number,types,and locations of brackets,and can correctly read the text of aircraft cables.The authoring tool can assist users who lack professional programming experience in establishing a process plan,i.e.,assembly outline based on AR for cable assembly.Conclusions The system can provide quick assembly guidance for aircraft cable with texts,images,and a 3 D model.It is beneficial for reducing the dependency on paper documents,labor intensity,and the error rate.

Integrated Virtual Assembly Process Planning System

Assembly process planning（APP） for complicated products is a time-consuming and difficult work with conventional method. Virtual assembly process planning（VAPP） provides engineers a new and efficiency way. Previous studies in VAPP are almost isolated and dispersive, and have not established a whole understanding and discussed key realization techniques of VAPP from a systemic and integrated view. The integrated virtual assembly process planning（IVAPP） system is a new virtual reality based engineering application, which offers engineers an efficient, intuitive, immersive and integrated method for assembly process planning in a virtual environment. Based on analysis the information integration requirement of VAPP, the architecture of IVAPP is proposed. Through the integrated structure, IVAPP system can realize information integration and workflow controlling. In order to mode/the assembly process in IVAPP, a hierarchical assembly task list（HATL） is presented, in which different assembly tasks for assembling different components are organized into a hierarchical list. A process-oriented automatic geometrical constraint recognition algorithm（AGCR） is proposed, so that geometrical constraints between components can be automatically recognized during the process of interactive assembling. At the same time, a progressive hierarchical reasoning（PHR） model is discussed. AGCR and PHR will greatly reduce the interactive workload. A discrete control node model（DCNM） for cable harness assembly planning in IVAPP is detailed. DCNM converts a cable harness into continuous flexed line segments connected by a series of section center points, and designs can realize cable harness planning through controlling those control nodes. Mechanical assemblies （such as transmission case and engine of automobile） are used to illustrate the feasibility of the proposed method and algorithms. The application of IVAPP system reveals advantages over the traditional assembly process planning method in shortening the time-consumed in assembly planning and in minimizing the handling difficulty, excessive reorientation and dissimilarity of assembly operations.