基于AR虚实图像注意力机制的电缆装配质量检测方法

赵甘霖; 余畅; 张建富; 杨建新; 冯平法; 沈群

doi:10.11959/j.issn.2096-3750.2021.00236

您当前的位置：

首页 >

文章列表页 >

基于AR虚实图像注意力机制的电缆装配质量检测方法

专题：工业互联网与智能制造 | 更新时间：2024-06-05

- 基于AR虚实图像注意力机制的电缆装配质量检测方法
- Inspection method for cable assembly quality based on AR virtual-real image attention mechanism
- 物联网学报 2021年5卷第3期页码：27-38
- 作者机构：
  
  1. 清华大学机械工程系，北京100084
  2. 清华大学基础工业训练中心，北京 100084
  3. 清华大学深圳国际研究生院先进制造学部，广东深圳 518055；4 航天东方红卫星有限公司，北京 100094
- 作者简介：
  
  [ "赵甘霖（1995- ），男，清华大学机械工程系博士生，主要研究方向为机器视觉、增强现实辅助装配技术等" ]
  [ "余畅（1998- ），男，清华大学机械工程系博士生，主要研究方向为机器视觉、增强现实、智能制造等" ]
  [ "张建富（1975- ），男，博士，清华大学机械工程系副教授、特别研究员，主要研究方向为精密加工技术、超声加工技术及系统、先进制造装备、智能制造系统等" ]
  [ "杨建新（1977- ），男，博士，清华大学基础工业训练中心副教授，主要研究方向为增强现实、智能制造、精密测量等" ]
  [ "冯平法（1966- ），男，博士，清华大学机械工程系教授，主要研究方向为智能制造与精密加工等" ]
  [ "沈群（1989- ），男，航天东方红卫星有限公司工程师，主要研究方向为卫星制造智能装备等" ]
- 基金信息：
  
  清华-东方红合作项目：基于AR的电缆网总装铺设引导系统软件开发;Tsinghua-DFH Cooperation Project: AR-based software development for cable assembly guidance system
- DOI：10.11959/j.issn.2096-3750.2021.00236
  中图分类号： TP391.9；V465
- 纸质出版日期：2021-09-30，
  
  网络出版日期：2021-09，
- 稿件说明：
移动端阅览
赵甘霖, 余畅, 张建富, 等. 基于AR虚实图像注意力机制的电缆装配质量检测方法[J]. 物联网学报, 2021,5(3):27-38.

GANLIN ZHAO, CHANG YU, JIANFU ZHANG, et al. Inspection method for cable assembly quality based on AR virtual-real image attention mechanism. [J]. Chinese journal on internet of things, 2021, 5(3): 27-38.
赵甘霖, 余畅, 张建富, 等. 基于AR虚实图像注意力机制的电缆装配质量检测方法[J]. 物联网学报, 2021,5(3):27-38. DOI： 10.11959/j.issn.2096-3750.2021.00236.

GANLIN ZHAO, CHANG YU, JIANFU ZHANG, et al. Inspection method for cable assembly quality based on AR virtual-real image attention mechanism. [J]. Chinese journal on internet of things, 2021, 5(3): 27-38. DOI： 10.11959/j.issn.2096-3750.2021.00236.

摘要

航天器中电缆装配在总装环节占据大量比重，电缆装配质量直接影响着产品整机的工作性能。现有的增强现实辅助电缆装配系统缺乏对装配结果实时判断和反馈的能力，无法识别当前装配的电缆是否符合装配质量要求。为了解决上述问题，提出了一种基于AR虚实图像注意力机制的电缆装配质量检测方法，通过构建虚拟辅助模型，在拍摄的装配结果图像中定位检测关键区域，过滤冗余图像信息；进而基于 YOLOv4-Tiny 图像检测模型对电缆装配关键节点位置进行了判断，基于计算电缆邻域平均重合度的方法求得了敷设路径的重合度，基于相机逆投影的方法得到了电缆的弯曲半径。最后，利用HoloLens2开发了基于增强现实技术的电缆装配质量检测系统，并对设计的4个电缆装配案例进行了质量检测，实验验证了本文方法及系统的可行性和有效性。

Abstract

Cable assembly in spacecraft occupies a large proportion in the final assembly process

and the quality of cable assembly directly affects the working performance of the whole product.The existing augmented reality-assisted cable assembly system lacks the ability to make real-time judgment and feedback on the assembly results

and cannot identify whether the currently assembled cable meets the quality requirements.In order to solve the above problems

a cable assembly quality inspection method based on AR virtual-real image attention mechanism was proposed.Critical areas in the captured images were located and detected by constructing virtual assist models to filter redundant image information.Based on YOLOv4-Tiny image detection model

the key node location of assembled cable was judged.The overlap of the laying path was obtained based on the method of calculating the average overlap of the neighborhood

and the bending radius of the cable was derived based on the method of camera inverse projection.Finally

an augmented reality-based cable assembly quality inspection system was developed using HoloLens2.The quality inspection was performed on the four designed cable assembly cases

and the feasibility and effectiveness of the method and system were experimentally verified.

关键词

电缆装配质量检测增强现实辅助装配AR虚实图像注意力机制

Keywords

cable assemblyquality inspectionaugmented reality assisted assemblyAR virtual-real imageattention mechanism

references

GOODDING J C, ARDELEAN E V, BABUSKA V ,et al. Experimental techniques and structural parameter estimation studies of spacecraft cables[J]. Journal of Spacecraft and Rockets, 2011,48(6): 942-957.

ARDELEAN E V, BABUŠKA V, GOODDING J C ,et al. Cable effects study:tangents,rabbit holes,dead ends,and valuable results[J]. Journal of Spacecraft and Rockets, 2014,52(2): 569-583.

WANG X, ONG S K, NEE A Y C . A comprehensive survey of augmented reality assembly research[J]. Advances in Manufacturing, 2016,4(1): 1-22.

CAUDELL T P, MIZELL D W . Augmented reality:an application of heads-up display technology to manual manufacturing processes[C]// Proceedings of the 25th Hawaii International Conference on System Sciences.[S.l.:s.n.], 1992,2: 659-669.

MIZELL D . Boeing's wire bundle assembly project[M]. Fundamentals of Wearable Computers and Augmented Reality. Boca Raton: CRC Press, 2001: 462-482.

SERVÁN J, MAS F, MENÉNDEZ J L ,et al. Using augmented reality in AIRBUS A400M shop floor assembly work instructions[C]// AIP Conference Proceedings. Melville:American Institute of Physics,2012, 1431（1）： 633-640.

ZHENG L Y, LIU X, AN Z W ,et al. A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection[J]. Virtual Reality ＆ Intelligent Hardware, 2020,2(1): 12-27.

ZHANG W, CHEN C Y, CHEN H J ,et al. Application of augmented reality in satellite cable network assembly[C]// IOP Conference Series:Materials Science and Engineering. Bristol:IOP Publishing, 2019,54(1): 012034.

陈华俊, 陈畅宇, 孙刚 ,等. 增强现实在航天器电缆装配中的应用技术研究[J]. 科技与创新, 2019(8): 40-44.

CHEN H J, CHEN C Y, SUN G ,et al. Research on the application technology of augmented reality in spacecraft cable assembly[J]. Science and Technology ＆ Innovation, 2019(8): 40-44.

PARMAR P, . Use of computer vision to detect tangles in tangled objects[C]// Proceedings of 2013 IEEE Second International Conference on Image Information Processing (ICIIP-2013). Piscataway:IEEE Press, 2013: 39-44.

SUN F J, WANG H, FAN J Q . 2D otsu segmentation algorithm based on simulated annealing genetic algorithm for iced-cable images[C]// Proceedings of 2009 International Forum on Information Technology and Applications. Piscataway:IEEE Press, 2009: 600-602.

温沛涵, 吴梦杰, 易树平 ,等. 基于目标识别的软管装配视觉检测系统[J]. 计算机集成制造系统, 2015,21(6): 1486-1496.

WEN P H, WU M J, YI S P ,et al. Visual inspection system for hose assembly based on object recognition[J]. Computer Integrated Manufacturing Systems, 2015,21(6): 1486-1496.

焦晨航 . 约束空间中电缆线安装方法研究[D]. 哈尔滨:哈尔滨工业大学, 2019.

JIAO C H . Research on cable installation method in constrained space[D]. Harbin:Harbin Institute of Technology, 2019.

刘睿, 尹旭悦, 范秀敏 ,等. 面向 AR 辅助引导的电缆敷设质量一致性视觉检测方法[J]. 计算机集成制造系统, 2021,27(1): 53-62.

LIU R, YIN X Y, FAN X M ,et al. AR guidance-based visual inspection method for cable laying quality consistency check[J]. Computer Integrated Manufacturing Systems, 2021,27(1): 53-62.

魏煌涛 . 基于视觉的电缆安装方法研究[D]. 哈尔滨:哈尔滨工业大学, 2020.

WEI H T . Research on installation method of cable based on vision[D]. Harbin:Harbin Institute of Technology, 2020.

冒文彦 . 基于卷积神经网络的高铁电缆线夹装配质量检测技术研究[D]. 镇江:江苏大学, 2020.

MAO W Y . Research on assembly quality inspection technology of high-speed rail cable clamps based on convolutional neural network[D]. Zhenjiang,China:Jiangsu University, 2020.

LI B, CHEN C . Transmission line detection based on a hierarchical and contextual model for aerial images[J]. Journal of Electronic Imaging, 2018,27(4): 1:043054.

LI S F, ZHENG P, ZHENG L Y . An AR-assisted deep learning-based approach for automatic inspection of aviation connectors[J]. IEEE Transactions on Industrial Informatics, 2021,17(3): 1721-1731.

BOCHKOVSKIY A, WANG C Y, LIAO H Y MARK . YOLOv4:optimal speed and accuracy of object detection[J]. arXiv:2004.10934, 2020.

EVERINGHAM M, WINN J . The PASCAL visual object classes challenge 2007 (voc2007) development kit[J]. International Journal of Computer Vision, 2006,111(1): 98-136.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

融合多尺度和上下文的无人机救援小目标检测算法

一种基于深度可分离卷积和注意力机制的入侵检测方法

基于空间特征的无线体域网人体姿态识别算法