Research progress of grating projection on machine 3D topography inspection technology
-
摘要:
基于视觉的测量方式对航天、军工以及电子芯片等先进制造领域具有良好应用前景以及深远的发展意义，而基于结构光的在机三维视觉检测技术，是目前精密加工领域的热点与难点之一。本文以结构光在机三维测量流程为主线，将其中的关键技术，包括测量标定、相位优化求解、在机三维点云处理及不同特征曲面重构中的技术要求、涉及的方法和原理、相关研究现状及目前存在的问题，进行论述与总结。最后，根据未来相关技术的实际需求，在加工现场标定、动态实时三维重构、亚微米及纳米级测量、测量-加工一体化数据传输技术等方面进行了展望，并提出了相应的研究思路。
Abstract:Vision-based measurement has good application prospects and far-reaching development significance for advanced manufacturing fields such as aerospace, the military industry and electronic chips. Among them, on-machine 3D vision detection technology based on structured light is one of the hotspots and challenges in the field of precision machining. Based on the on-machine 3D measurement process of structured light, we discuss and summarize the key technologies, including its technical requirements, methods and principles involved, related research status and existing problems in the measurement calibration, phase optimization solution, on-machine 3D point cloud processing and reconstruction of different feature surfaces. Finally, according to the actual needs of relevant technologies in the future, prospects are made with regard to processing field calibration, dynamic real-time 3D reconstruction, sub-micron and nano measurement, and measurement processing integrated data transmission technology, with the corresponding research ideas put forward.
-
图 2 基于结构光的在机检测系统架构
Figure 2. Architecture of an on-machine detection system based on structured light
图 4 本课题组研发的单目结构光智能在机检测平台
Figure 4. Monocular structured light intelligent on-machine detection platform developed by our research group
图 13 基于GA-BP算法的点云孔洞修补流程
Figure 13. Point cloud hole repair process based on the GA-BP algorithm
表 1 光学测量系统平台性能及特点
Table 1. Performance and characteristics of the optical measurement system platform
测量系统平台 测量精度 距离量程 优点 缺点 适用场景 电子经纬仪 10 μm 0~150 m 具有误差自修正功能，
抗干扰性强内部元器件制造
误差制约测量精度大尺寸零件装配
及尺寸检测结构光投影测量系统 0.1~20 μm 0.1~10 m 非逐点检测，测量效率高 测量过程受反光影响 复杂形貌零件，高效测量 扫描仪 0.1~10 μm 0~80 m 测量精度高，便携性好 测量效率较低 小尺寸零件，动态测量 机械臂测量系统 50 μm 0~5 m 自动化程度高 场景约束性强，便携性较差 全自动化加工检测场景 白光干涉仪 纳米级或亚纳米级 150 μm~20 mm 测量精度较高 使用条件及要求较苛刻，
测量效率低超精密加工及3C电子检测 
下载: 导出CSV
表 3 典型的光饱和优化方法对比
Table 3. Comparison of typical optical saturation optimization methods
代表性学者 技术方法 优点 缺点 适用场景 Pinzek [29] 多重曝光控制法 曝光参数控制较方便，无需额外硬件系统 难以根据环境定量精确控制曝光，
耗时长环境光照影响较为主要时的检测 Liu [32] 全自动快速最佳曝光计算方法 检测过程自动化水平
较高易影响反射率较小部位，造成曝光
不足表面尺寸较大的反光物体检测 陈龙[35] 局部自适应条纹投影法 具有针对性的处理，
不影响其他区域像素准确的灰度阈值较难
确定叶片、轴类表面反射率
差异大的对象检测Liu [37] 多目视点配准法 测量精度
较高受测量空间限制，欠缺灵活性 小范围精密物体的在机检测 Zhang [38] 偏振滤光
片法可较精确控制光平衡 需要添加额外的光学及控制硬件 镜面物体的在机检测
下载: 导出CSV
表 4 在机动态相位补偿代表性方法对比
Table 4. Comparison of representative methods of on machine dynamic phase compensation
下载: 导出CSV
表 5 常用的离群点去除方式
Table 5. Common outlier removal methods
离群点去除方法 适用场景 基于迭代优化算法 适合于在杂乱噪声中搜寻临近目标点 基于邻域信息的
点云聚类算法适合对具有特定类型的
点云数据进行分类基于点云频率的滤波方式 适合对较复杂形状点云模型的分割计算
下载: 导出CSV
表 6 不同类型曲面形貌重构技术对比
Table 6. Comparison of different types of surface reconstruction technologies
曲面类型 典型代表 曲面特点 测量难点 适用测量策略 曲面拟合方法 大尺寸曲面零件 涡轮发动机叶片、航空曲面零件 形状较规律，曲面尺寸数米至数十米 一次扫描难以测量出完整表面点云，且数据计算量庞大 采用柔性测量装置并配合多视角点云拼接方式 曲面片直接
拟合法微型曲面零件 微型精密半导体零件、精密光学器件 曲面尺寸较微小
(毫米级)点云分割难度较大，且较难解决
过拟合和欠拟合问题搭载光学高倍镜头和立体显微镜的光栅条纹投影法的测量方式 样条曲线
拟合方法复杂形状及曲面结合体 复杂腔体类零件 结构复杂，面数较多 点云模型拟合困难，测量繁琐，难以获得高精度且完整的点云模型 接触式与非接触式结构
光结合的测量方式基于特征约束及交互式曲面拟合方法
下载: 导出CSV
-
[1] DE-DIN. BS EN ISO 25178-6-2010 Geometrical product specifications (GPS) - surface texture: areal - part 6: classification of methods for measuring surface texture[S]. 2010. [2] 中华人民共和国国家质量监督检验检疫总局, 中国国家标准化管理委员会. GB/T 33523.6-2017 产品几何技术规范(GPS) 表面结构 区域法 第6部分: 表面结构测量方法的分类[S]. 北京: 中国标准出版社, 2017.General Administration of Quality Supervision, Inspection and Quarantine of the People's Republic of China, Standardization Administration. GB/T 33523.6-2017 Geometrical product specifications(GPS)—surface texture: areal—part 6: classification of methods for measuring surface texture[S]. Beijing: Standards Press of China, 2017. (in Chinese) [3] WANG Z P, YIN M, OU D Y, et al. Optical measurement method for blade profiles based on blade self-features[J]. IEEE Transactions on Industrial Electronics, 2022, 69(2): 2067-2076. doi: 10.1109/TIE.2021.3062213 [4] ZHENG ZH, WAN Y, ZHANG Y J, et al. CLNet: cross-layer convolutional neural network for change detection in optical remote sensing imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 175: 247-267. doi: 10.1016/j.isprsjprs.2021.03.005 [5] ZANGL K, DANZL R, HELMLI F, et al. Highly accurate optical µCMM for measurement of micro holes[J]. Procedia CIRP, 2018, 75: 397-402. doi: 10.1016/j.procir.2018.05.098 [6] KHAN A, MINEO C, DOBIE G, et al. Vision guided robotic inspection for parts in manufacturing and remanufacturing industry[J]. Journal of Remanufacturing, 2021, 11(1): 49-70. doi: 10.1007/s13243-020-00091-x [7] LIU J D, SUN W L, HUANG Y. An algorithm for trajectory planning of complex surface parts for laser cladding remanufacturing[J]. Proceedings of the Institution of Mechanical Engineers,Part B:Journal of Engineering Manufacture, 2021, 235(12): 2025-2032. doi: 10.1177/0954405420987712 [8] 陶迁, 周志峰, 吴明晖, 等. 基于相位测量偏折术的反射表面缺陷检测[J]. 液晶与显示,2020,35(12):1315-1322. doi: 10.37188/YJYXS20203512.1315TAO Q, ZHOU ZH F, WU M H, et al. Detection of reflective surface defects based on phase measuring deflectometry[J]. Chinese Journal of Liquid Crystals and Displays, 2020, 35(12): 1315-1322. (in Chinese) doi: 10.37188/YJYXS20203512.1315 [9] REVILLA-LEÓN M, GONZALEZ-MARTÍN Ó, PÉREZ LÓPEZ J, et al. Position accuracy of implant analogs on 3D printed polymer versus conventional dental stone casts measured using a coordinate measuring machine[J]. Journal of Prosthodontics, 2018, 27(6): 560-567. doi: 10.1111/jopr.12708 [10] 朱可, 霍彦文, 武通海, 等. 基于光度立体视觉三维重构算法的微观磨损形貌原位测量原理及方法[J]. 机械工程学报,2021,57(10):1-9. doi: 10.3901/JME.2021.10.001ZHU K, HUO Y W, WU T H, et al. Principle and method for in-situ measurement of micro-scale worn surface morphology based on 3D reconstruction with photometric stereo vision Algorithm[J]. Journal of Mechanical Engineering, 2021, 57(10): 1-9. (in Chinese) doi: 10.3901/JME.2021.10.001 [11] 李中伟, 张攀, 钟凯, 等. AutoScan系列复杂零件自动化三维测量装备开发与应用[J]. 航空学报,2021,42(10):112-129.LI ZH W, ZHANG P, ZHONG K, et al. Development and application of AutoScan series automated 3D measuring equipment for complex parts[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(10): 112-129. (in Chinese) [12] 邱志成, 肖骏. 线结构光视觉系统标定新法及其振动测控应用[J]. 光学 精密工程,2019,27(1):230-240. doi: 10.3788/OPE.20192701.0230QIU ZH CH, XIAO J. New calibration method of line structured light vision system and application for vibration measurement and control[J]. Optics and Precision Engineering, 2019, 27(1): 230-240. (in Chinese) doi: 10.3788/OPE.20192701.0230 [13] 李茂月, 肖桂风, 蔡东辰, 等. 单目结构光测量中参数自适应标定方法[J/OL]. 红外与 工程 [2022-03-21]. http://kns.cnki.net/kcms/detail/12.1261.tn.20211207.0957.002.html.LI M Y, XIAO G F, CAI D CH, et al. . Parameter adaptive calibration method in monocular structured light measurement[J/OL]. Infrared and Laser Engineering [2022-03-21]. http://kns.cnki.net/kcms/detail/12.1261.tn.20211207.0957.002.html. (in Chinese) [14] TAKEDA M, MUTOH K. Fourier transform profilometry for the automatic measurement of 3-D object shapes[J]. Applied Optics, 1983, 22(24): 3977-3982. doi: 10.1364/AO.22.003977 [15] WU ZH J, GUO W B, LU L L, et al. Generalized phase unwrapping method that avoids jump errors for fringe projection profilometry[J]. Optics Express, 2021, 29(17): 27181-27192. doi: 10.1364/OE.436116 [16] GUAN F L, XU A L, JIANG G Y, et al. An improved fast camera calibration method for mobile terminals[J]. Journal of Information Processing Systems, 2019, 15(5): 1082-1095. [17] YAO Q, KUBOTA A, KAWAKITA K, et al. . Fast camera self-calibration for synthesizing free viewpoint soccer video[C]. Proceedings of 2017 IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, 2017: 1612-1616. [18] GAO Z R, GAO Y, SU Y, et al. Stereo camera calibration for large field of view digital image correlation using zoom lens[J]. Measurement, 2021, 185: 109999. doi: 10.1016/j.measurement.2021.109999 [19] ZHANG J, LUO B, XIANG ZH L, et al. Deep-learning-based adaptive camera calibration for various defocusing degrees[J]. Optics Letters, 2021, 46(22): 5537-5540. doi: 10.1364/OL.443337 [20] XU X CH, LIU M Y, PENG S, et al. An In-orbit stereo navigation camera self-calibration method for planetary rovers with multiple constraints[J]. Remote Sensing, 2022, 14(2): 402. doi: 10.3390/rs14020402 [21] FOLTYNSKI P, LADYZYNSKI P. Digital planimetry with a new adaptive calibration procedure results in accurate and precise wound area measurement at curved surfaces[J]. Journal of Diabetes Science and Technology, 2022, 16(1): 128-136. doi: 10.1177/1932296820959346 [22] MORU D K, BORRO D. Analysis of different parameters of influence in industrial cameras calibration processes[J]. Measurement, 2021, 171: 108750. doi: 10.1016/j.measurement.2020.108750 [23] 管雯璐, 谭逢富, 靖旭, 等. 基于环境温度反馈的卡塞格林望远镜自动调焦[J]. 光学 精密工程,2021,29(8):1832-1838. doi: 10.37188/OPE.20212908.1832GUAN W L, TAN F F, JING X, et al. Automatic focusing of Cassegrain telescope based on environmental temperature feedback[J]. Optics and Precision Engineering, 2021, 29(8): 1832-1838. (in Chinese) doi: 10.37188/OPE.20212908.1832 [24] 潘海鸿, 王耀玮, 徐大斌, 等. 使用非特制靶标实现线结构光平面标定[J]. 光学 精密工程, 2021, 29(1): 54-60.PAN H H, WANG Y W, XU D B, et al.. Calibration of line-structure light plane using non-specific target[J] Optics and Precision Engineering, 2021, 29(1): 54-60. (in Chinese) [25] CHOI H, RYU J. Design of wide angle and large aperture optical system with inner focus for compact system camera applications[J]. Applied Sciences, 2019, 10(1): 179. [26] PAK I J, KIM C S, KANG J C, et al. Verification of phase measurement error sources in phase-shifting interferometry with four step phase-shifting algorithms[J]. Applied Optics, 2021, 60(13): 3856-3864. doi: 10.1364/AO.418495 [27] LI J, GUAN J T, DU H, et al. Error self-correction method for phase jump in multi-frequency phase-shifting structured light[J]. Applied Optics, 2021, 60(4): 949-958. doi: 10.1364/AO.413506 [28] LIN H, GAO J, MEI Q, et al. Adaptive digital fringe projection technique for high dynamic range three-dimensional shape measurement[J]. Optics Express, 2016, 24(7): 7703-7718. doi: 10.1364/OE.24.007703 [29] PINZEK S, GUSTSCHIN A, NEUWIRTH T, et al. Signal retrieval from non-sinusoidal intensity modulations in X-ray and neutron interferometry using piecewise-defined polynomial function[J]. Journal of Imaging, 2021, 7(10): 209. doi: 10.3390/jimaging7100209 [30] RAO L, DA F P. High dynamic range 3D shape determination based on automatic exposure selection[J]. Journal of Visual Communication and Image Representation, 2018, 50: 217-226. doi: 10.1016/j.jvcir.2017.12.003 [31] 李茂月, 刘泽隆, 赵伟翔, 等. 面结构光在机检测的叶片反光抑制技术[J]. 中国光学,2022,15(3):464-475. doi: 10.37188/CO.2021-0194LI M Y, LIU Z L, ZHAO W X, et al. Blade reflection suppression technology based on surface structured light on-machine detection[J]. Chinese Optics, 2022, 15(3): 464-475. (in Chinese) doi: 10.37188/CO.2021-0194 [32] LIU X J, CHEN W Y, MADHUSUDANAN H, et al. Optical measurement of highly reflective surfaces from a single exposure[J]. IEEE Transactions on Industrial Informatics, 2021, 17(3): 1882-1891. doi: 10.1109/TII.2020.2991458 [33] WANG YH, ZHANG Q, HU Y, et al. Rapid 3D measurement of high dynamic range surface based on multi-polarization fringe projection[J]. Optical Engineering, 2021, 60(8): 084107. [34] RAMAKRISHNAN V, PETE D J. Savitzky–Golay filtering-based fusion of multiple exposure images for high dynamic range imaging[J]. SN Computer Science, 2021, 2(3): 191. doi: 10.1007/s42979-021-00594-9 [35] 陈龙, 王文聪, 张峰峰, 等. 基于双目结构光的术中肝脏表面局部亮度饱和分区投影[J]. 光学 精密工程, 2021, 29(11): 2590-2602.CHEN L, WANG W C, ZHANG F F, et al. . Zonal projection based on binocular structured light for localized luminance saturation of intraoperative liver surface[J] Optics and Precision Engineering, 2021, 29(11): 2590-2602. (in Chinese) [36] JIANG H ZH, ZHAO H J, LI X D. High dynamic range fringe acquisition: a novel 3-D scanning technique for high-reflective surfaces[J]. Optics and Lasers in Engineering, 2012, 50(10): 1484-1493. doi: 10.1016/j.optlaseng.2011.11.021 [37] LIU G H, LIU X Y, FENG Q Y. 3D shape measurement of objects with high dynamic range of surface reflectivity[J]. Applied Optics, 2011, 50(23): 4557-4565. doi: 10.1364/AO.50.004557 [38] ZHANG B L, KWOK H S, HUANG H C. Three-dimensional optical modeling and optimizations of color filter liquid-crystal-on-silicon microdisplays[J]. Journal of Applied Physics, 2005, 98(12): 123103. doi: 10.1063/1.2149494 [39] CHEN M Y, GUO H W, WEI C L. Algorithm immune to tilt phase-shifting error for phase-shifting interferometers[J]. Applied Optics, 2000, 39(22): 3894-3898. doi: 10.1364/AO.39.003894 [40] FENG SH J, ZUO CH, TAO T Y, et al. Robust dynamic 3-D measurements with motion-compensated phase-shifting profilometry[J]. Optics and Lasers in Engineering, 2018, 103: 127-138. doi: 10.1016/j.optlaseng.2017.12.001 [41] DENG J, LI J, FENG H, et al. Edge-preserved fringe-order correction strategy for code-based fringe projection profilometry[J]. Signal Processing, 2021, 182: 107959. doi: 10.1016/j.sigpro.2020.107959 [42] 郭逸凡. 运动物体散焦光栅投影三维测量技术研究[D]. 南京: 东南大学, 2019: 30-42.GUO Y F. The research of defocusing fringe pattern projection profilometry on dynamic objects[D]. Nanjing: Southeast University, 2019: 30-42. (in Chinese) [43] KIM S S, CHUNG B M. Rapid 3D shape measurement using sine pattern in phase-shifting projection method[J]. International Journal of Precision Engineering and Manufacturing, 2021, 22(8): 1381-1389. doi: 10.1007/s12541-021-00545-7 [44] WEISE T, LEIBE B, VAN GOOL L. Fast 3D scanning with automatic motion compensation[C]. Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2007: 1-8. [45] TAO T Y, CHEN Q, FENG SH J, et al. High-speed real-time 3D shape measurement based on adaptive depth constraint[J]. Optics Express, 2018, 26(17): 22440-22456. doi: 10.1364/OE.26.022440 [46] GONG M, ZHANG ZH J, ZENG D. A new simplification algorithm for scattered point clouds with feature preservation[J]. Symmetry, 2021, 13(3): 399. doi: 10.3390/sym13030399 [47] 杜钦生, 李丹丹, 陈浩, 等. 结构光3D点云的PIN针针尖提取[J]. 液晶与显示,2021,36(9):1331-1340. doi: 10.37188/CJLCD.2020-0321DU Q SH, LI D D, CHEN H, et al. PIN tip extraction from 3D point cloud of structured light[J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36(9): 1331-1340. (in Chinese) doi: 10.37188/CJLCD.2020-0321 [48] NIE E J, LI J, ZHANG R M, et al. . Three dimensional point cloud hole repairing strategy for binocular stereo reconstruction[C]. Proceedings of 2017 IEEE International Conference on Robotics and Biomimetics, IEEE, 2017: 2456-2461. [49] JU T. Robust repair of polygonal models[J]. ACM Transactions on Graphics, 2004, 23(3): 888-895. doi: 10.1145/1015706.1015815 [50] CHEN H, CUI W. Holes filling of scattered point cloud based on simplification[J]. Multimedia Tools and Applications, 2022, 81(11): 14641-14661. doi: 10.1007/s11042-021-11019-3 [51] CHU T, YAO W M, LIU J, et al. Hole-filling framework by combining structural and textural information for the 3D Terracotta Warriors[J]. Journal of Applied Remote Sensing, 2021, 15(4): 046503. [52] 李月雯, 耿国华, 魏潇然. 基于泊松方程的孔洞修补算法[J]. 计算机工程,2017,43(10):209-215,221. doi: 10.3969/j.issn.1000-3428.2017.10.035LI Y W, GENG G H, WEI X R. Hole-filling algorithm based on Poisson equation[J]. Computer Engineering, 2017, 43(10): 209-215,221. (in Chinese) doi: 10.3969/j.issn.1000-3428.2017.10.035 [53] 刘中玉, 张明锋, 聂雪媛, 等. 一种基于径向基函数的两步法网格变形策略[J]. 力学学报,2015,47(3):534-538. doi: 10.6052/0459-1879-14-280LIU ZH Y, ZHANG M F, NIE X Y, et al. A two-step mesh deformation strategy based on radial basis function[J]. Chinese Journal of Theoretical and Applied Mechanics, 2015, 47(3): 534-538. (in Chinese) doi: 10.6052/0459-1879-14-280 [54] LIN H B, WANG W. Feature preserving holes filling of scattered point cloud based on tensor voting[C]. Proceedings of 2016 IEEE International Conference on Signal and Image Processing, IEEE, 2016: 402-406. [55] ZHANG H, TIAN ZH G. Failure analysis of corroded high-strength pipeline subject to hydrogen damage based on FEM and GA-BP neural network[J]. International Journal of Hydrogen Energy, 2022, 47(7): 4741-4758. doi: 10.1016/j.ijhydene.2021.11.082 [56] WANG J SH, GONG Z Y, TAO B, et al. A 3-D reconstruction method for large freeform surfaces based on mobile robotic measurement and global optimization[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 5006809. [57] WANG J SH, TAO B, GONG Z Y, et al. A mobile robotic measurement system for large-scale complex components based on optical scanning and visual tracking[J]. Robotics and Computer-Integrated Manufacturing, 2021, 67: 102010. doi: 10.1016/j.rcim.2020.102010 [58] GUO B Y, WANG J W, JIANG X B, et al. A 3D surface reconstruction method for large-scale point cloud data[J]. Mathematical Problems in Engineering, 2020, 2020: 8670151. [59] DE PAOLIS L T, DE LUCA V, GATTO C, et al. . Photogrammetric 3D reconstruction of small objects for a real-time fruition[C]. Proceedings of the 7th International Conference on Augmented Reality, Virtual Reality, and Computer Graphics, Springer, 2020: 375-394. [60] 张溪溪, 纪小刚, 胡海涛, 等. 基于特征线拟合的微型复杂曲面点云分割方法[J]. 与光电子学进展,2020,57(6):061502.ZHANG X X, JI X G, HU H T, et al. Point cloud segmentation method for complex micro-surface based on feature line fitting[J]. Laser &Optoelectronics Progress, 2020, 57(6): 061502. (in Chinese) [61] 吴恩启, 柯映林, 李江雄. 微细管道内表面光电检测及三维重构系统[J]. 光电工程,2007,34(1):59-64.WU E Q, KE Y L, LI J X. Photoelectric detection and 3D reconstruction system for the inner surface of small-diameter pipes[J]. Opto-Electronic Engineering, 2007, 34(1): 59-64. (in Chinese) [62] HARIYAMA T, MARUNO K, WATANABE M, et al. High-accuracy error-reduction method for 3D complex shape measurement with local-maximum-power-based technique of FMCW[J]. Precision Engineering, 2021, 72: 69-82. doi: 10.1016/j.precisioneng.2021.04.004 -
下载:
点击查看大图
图(17) / 表(6)
计量
- 文章访问数: 397
- HTML全文浏览量: 245
- PDF下载量: 367
- 被引次数: 0
