Inspection method for cable assembly quality based on AR virtual-real image attention mechanism

Ganlin ZHAO; Chang YU; Jianfu ZHANG; Jianxin YANG; Pingfa FENG; Qun SHEN

doi:10.11959/j.issn.2096-3750.2021.00236

您当前的位置：

首页 >

文章列表页 >

Inspection method for cable assembly quality based on AR virtual-real image attention mechanism

Topic: Industrial Internet and Smart Manufacturing | 更新时间：2024-06-05

- Inspection method for cable assembly quality based on AR virtual-real image attention mechanism
- Chinese Journal on Internet of Things Vol. 5, Issue 3, Pages: 27-38(2021)
- 作者机构：
  
  1. 清华大学机械工程系，北京100084
  2. 清华大学基础工业训练中心，北京 100084
  3. 清华大学深圳国际研究生院先进制造学部，广东深圳 518055；4 航天东方红卫星有限公司，北京 100094
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-3750.2021.00236
  CLC： TP391.9；V465
- Published：30 September 2021，
  
  Published Online：2021-09，
- 稿件说明：
移动端阅览
GANLIN ZHAO, CHANG YU, JIANFU ZHANG, et al. Inspection method for cable assembly quality based on AR virtual-real image attention mechanism. [J]. Chinese journal on internet of things, 2021, 5(3): 27-38.
DOI：

GANLIN ZHAO, CHANG YU, JIANFU ZHANG, et al. Inspection method for cable assembly quality based on AR virtual-real image attention mechanism. [J]. Chinese journal on internet of things, 2021, 5(3): 27-38. DOI： 10.11959/j.issn.2096-3750.2021.00236.

摘要

航天器中电缆装配在总装环节占据大量比重，电缆装配质量直接影响着产品整机的工作性能。现有的增强现实辅助电缆装配系统缺乏对装配结果实时判断和反馈的能力，无法识别当前装配的电缆是否符合装配质量要求。为了解决上述问题，提出了一种基于AR虚实图像注意力机制的电缆装配质量检测方法，通过构建虚拟辅助模型，在拍摄的装配结果图像中定位检测关键区域，过滤冗余图像信息；进而基于 YOLOv4-Tiny 图像检测模型对电缆装配关键节点位置进行了判断，基于计算电缆邻域平均重合度的方法求得了敷设路径的重合度，基于相机逆投影的方法得到了电缆的弯曲半径。最后，利用HoloLens2开发了基于增强现实技术的电缆装配质量检测系统，并对设计的4个电缆装配案例进行了质量检测，实验验证了本文方法及系统的可行性和有效性。

Abstract

Cable assembly in spacecraft occupies a large proportion in the final assembly process

and the quality of cable assembly directly affects the working performance of the whole product.The existing augmented reality-assisted cable assembly system lacks the ability to make real-time judgment and feedback on the assembly results

and cannot identify whether the currently assembled cable meets the quality requirements.In order to solve the above problems

a cable assembly quality inspection method based on AR virtual-real image attention mechanism was proposed.Critical areas in the captured images were located and detected by constructing virtual assist models to filter redundant image information.Based on YOLOv4-Tiny image detection model

the key node location of assembled cable was judged.The overlap of the laying path was obtained based on the method of calculating the average overlap of the neighborhood

and the bending radius of the cable was derived based on the method of camera inverse projection.Finally

an augmented reality-based cable assembly quality inspection system was developed using HoloLens2.The quality inspection was performed on the four designed cable assembly cases

and the feasibility and effectiveness of the method and system were experimentally verified.

关键词

电缆装配质量检测增强现实辅助装配AR虚实图像注意力机制

Keywords

cable assemblyquality inspectionaugmented reality assisted assemblyAR virtual-real imageattention mechanism

references

GOODDING J C, ARDELEAN E V, BABUSKA V ,et al. Experimental techniques and structural parameter estimation studies of spacecraft cables[J]. Journal of Spacecraft and Rockets, 2011,48(6): 942-957.

ARDELEAN E V, BABUŠKA V, GOODDING J C ,et al. Cable effects study:tangents,rabbit holes,dead ends,and valuable results[J]. Journal of Spacecraft and Rockets, 2014,52(2): 569-583.

WANG X, ONG S K, NEE A Y C . A comprehensive survey of augmented reality assembly research[J]. Advances in Manufacturing, 2016,4(1): 1-22.

CAUDELL T P, MIZELL D W . Augmented reality:an application of heads-up display technology to manual manufacturing processes[C]// Proceedings of the 25th Hawaii International Conference on System Sciences.[S.l.:s.n.], 1992,2: 659-669.

MIZELL D . Boeing's wire bundle assembly project[M]. Fundamentals of Wearable Computers and Augmented Reality. Boca Raton: CRC Press, 2001: 462-482.

SERVÁN J, MAS F, MENÉNDEZ J L ,et al. Using augmented reality in AIRBUS A400M shop floor assembly work instructions[C]// AIP Conference Proceedings. Melville:American Institute of Physics,2012, 1431（1）： 633-640.

ZHENG L Y, LIU X, AN Z W ,et al. A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection[J]. Virtual Reality ＆ Intelligent Hardware, 2020,2(1): 12-27.

ZHANG W, CHEN C Y, CHEN H J ,et al. Application of augmented reality in satellite cable network assembly[C]// IOP Conference Series:Materials Science and Engineering. Bristol:IOP Publishing, 2019,54(1): 012034.

陈华俊, 陈畅宇, 孙刚 ,等. 增强现实在航天器电缆装配中的应用技术研究[J]. 科技与创新, 2019(8): 40-44.

CHEN H J, CHEN C Y, SUN G ,et al. Research on the application technology of augmented reality in spacecraft cable assembly[J]. Science and Technology ＆ Innovation, 2019(8): 40-44.

PARMAR P, . Use of computer vision to detect tangles in tangled objects[C]// Proceedings of 2013 IEEE Second International Conference on Image Information Processing (ICIIP-2013). Piscataway:IEEE Press, 2013: 39-44.

SUN F J, WANG H, FAN J Q . 2D otsu segmentation algorithm based on simulated annealing genetic algorithm for iced-cable images[C]// Proceedings of 2009 International Forum on Information Technology and Applications. Piscataway:IEEE Press, 2009: 600-602.

温沛涵, 吴梦杰, 易树平 ,等. 基于目标识别的软管装配视觉检测系统[J]. 计算机集成制造系统, 2015,21(6): 1486-1496.

WEN P H, WU M J, YI S P ,et al. Visual inspection system for hose assembly based on object recognition[J]. Computer Integrated Manufacturing Systems, 2015,21(6): 1486-1496.

焦晨航 . 约束空间中电缆线安装方法研究[D]. 哈尔滨:哈尔滨工业大学, 2019.

JIAO C H . Research on cable installation method in constrained space[D]. Harbin:Harbin Institute of Technology, 2019.

刘睿, 尹旭悦, 范秀敏 ,等. 面向 AR 辅助引导的电缆敷设质量一致性视觉检测方法[J]. 计算机集成制造系统, 2021,27(1): 53-62.

LIU R, YIN X Y, FAN X M ,et al. AR guidance-based visual inspection method for cable laying quality consistency check[J]. Computer Integrated Manufacturing Systems, 2021,27(1): 53-62.

魏煌涛 . 基于视觉的电缆安装方法研究[D]. 哈尔滨:哈尔滨工业大学, 2020.

WEI H T . Research on installation method of cable based on vision[D]. Harbin:Harbin Institute of Technology, 2020.

冒文彦 . 基于卷积神经网络的高铁电缆线夹装配质量检测技术研究[D]. 镇江:江苏大学, 2020.

MAO W Y . Research on assembly quality inspection technology of high-speed rail cable clamps based on convolutional neural network[D]. Zhenjiang,China:Jiangsu University, 2020.

LI B, CHEN C . Transmission line detection based on a hierarchical and contextual model for aerial images[J]. Journal of Electronic Imaging, 2018,27(4): 1:043054.

LI S F, ZHENG P, ZHENG L Y . An AR-assisted deep learning-based approach for automatic inspection of aviation connectors[J]. IEEE Transactions on Industrial Informatics, 2021,17(3): 1721-1731.

BOCHKOVSKIY A, WANG C Y, LIAO H Y MARK . YOLOv4:optimal speed and accuracy of object detection[J]. arXiv:2004.10934, 2020.

EVERINGHAM M, WINN J . The PASCAL visual object classes challenge 2007 (voc2007) development kit[J]. International Journal of Computer Vision, 2006,111(1): 98-136.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Fusion of multi-scale and context for small target detection algorithm of unmanned aerial vehicle rescue

An intrusion detection method based on depthwise separable convolution and attention mechanism

Human activity recognition algorithm based on the spatial feature for WBAN

Related Author

LIU Yuan

ZHAO Jing

JIANG Guoping

XU Fengyu

LU Ningyun

Zhifei ZHANG

Feng LIU

Yiyang GE

Related Institution

College of Automation & College of Artificial Intelligence, Nanjing University of Post and Telecommunication

State Key Laboratory of Mechanics and Control for Aerospace Structures, Nanjing University of Aeronautics and Astronautics

Engineering Research Center of Network Management Technology for High Speed Railway of Ministry of Education, School of Computer and Information Technology, Beijing Jiaotong University

Collaborative Innovation Center of Railway Traffic Safety, Beijing Jiaotong University

National Engineering Research Center of Advanced Network Technologies, Beijing Jiaotong University

⁰