May 2021

Article Contents

Chinese Optics> 2021> 14(3): 447-457.

WANG Yong-hong, ZHANG Qian, HU Yin, WANG Huan-qing. 3D small-field surface imaging based on microscopic fringe projection profilometry：a review[J]. Chinese Optics, 2021, 14(3): 447-457. doi: 10.37188/CO.2020-0199

Citation:

WANG Yong-hong, ZHANG Qian, HU Yin, WANG Huan-qing. 3D small-field surface imaging based on microscopic fringe projection profilometry：a review[J].Chinese Optics, 2021, 14(3): 447-457.doi:10.37188/CO.2020-0199

Citation:

WANG Yong-hong, ZHANG Qian, HU Yin, WANG Huan-qing. 3D small-field surface imaging based on microscopic fringe projection profilometry：a review[J].Chinese Optics, 2021, 14(3): 447-457.doi:10.37188/CO.2020-0199

PDF( 2289 KB)

3D small-field surface imaging based on microscopic fringe projection profilometry：a review

doi:10.37188/CO.2020-0199

School of Instrument Science and Opto-electronics Engineering, Hefei University of Technology, Hefei 230009, China

Funds:Supported by National Key Research and Development Program of China (No. 2016YFF0101803); National Natural Science Foundation of China (No. 51805137)

More Information

Corresponding author:yhwang@hfut.edu.cn
Received Date:10 Nov 2020
Rev Recd Date:07 Jan 2021

Available Online:27 Mar 2021

Publish Date:14 May 2021

Abstract

Abstract

Intelligent manufacturing has become more precise, miniaturized and integrated. Representative integrated circuit technology and its derived miniature sensors such as Micro-Electro-Mechanical System (MEMS) have become widely used. Therefore, it is important for intelligent manufacturing development to accurately obtain the surface morphology information of micro-devices and implement rapid detection of device surface defects. Fringe Projection Profilometry (FPP) based on structural light projection has the advantages of being non-contact, highly precise, highly efficient and having full-field measurement, which plays an important role in the field of precision measurement. Microscopic Fringe Projection Profilometry (MFPP) has been developed rapidly during recent decades. In recent years, MFPP has made great progress in many aspects, including its optical system structures, corresponding system calibration methods, phase extraction algorithms, and 3D coordinate reconstruction methods. In this paper, the structure and principle of a three-dimensional measurement system of microscopic fringe projection are reviewed, and the calibration problem of a small field-of-view system that is different from the traditional projection model is analyzed. After that, the development and improvement process of the micro-projection system structure is introduced, and the reflection in the measurment caused by the system structure and metal material is analyzed. On this basis, the prospects of the development of microscopic fringe projection of 3D measurement system are discussed.
- small visual field measurement,
- 3D reconstruction,
- microscopic fringe projection profilometry,
- high dynamic range technology,
- calibration

FullText(HTML)

References (60)

References

[1]	WANG J H, YANG Y X, SHAO M W, et al. Three-dimensional measurement for rigid moving objects based on multi-fringe projection[J]. IEEE Photonics Journal, 2020, 12(4): 6802114.
[2]	XIA P, WANG Q H, RI SH E. Random phase-shifting digital holography based on a self-calibrated system[J]. Optics Express, 2020, 28(14): 19988-19996. doi:10.1364/OE.395819
[3]	屈铭, 郑俊杰, 李敏, 等. 基于扫描白光干涉法的LCOS芯片像素级相位分析[J]. 光子学报,2019,48(9):0911004. doi:10.3788/gzxb20194809.0911004 QU M, ZHENG J J, LI M, et al. Pixel-level observation of phase profile in liquid crystal on silicon device by white light scanning interferometry[J]. Acta Photonica Sinica, 2019, 48(9): 0911004. (in Chinese) doi:10.3788/gzxb20194809.0911004
[4]	MURAKAMI H, KATSUKI A, SAJIMA T, et al. Investigation of factors affecting sensitivity enhancement of an optical fiber probe for microstructure measurement using oblique incident light[J]. Applied Sciences, 2020, 10(9): 3191. doi:10.3390/app10093191
[5]	李成辉, 田云飞, 闫曙光. 扫描共聚焦显微成像技术与应用[J]. 实验科学与技术,2020,18(4):33-38. doi:10.12179/1672-4550.20190257 LI CH H, TIAN Y F, YAN SH G. Laser scanning confocal microscopy and its application[J]. Experiment Science and Technology, 2020, 18(4): 33-38. (in Chinese) doi:10.12179/1672-4550.20190257
[6]	HU Y, CHEN Q, FENG SH J, et al. Microscopic fringe projection profilometry: a review[J]. Optics and Lasers in Engineering, 2020, 135: 106192. doi:10.1016/j.optlaseng.2020.106192
[7]	LEONHARDT K, DROSTE U, TIZIANI H J. Microshape and rough-surface analysis by fringe projection[J]. Applied Optics, 1994, 33(31): 7477-7488. doi:10.1364/AO.33.007477
[8]	QUAN C, HE X Y, WANG C F, et al. Shape measurement of small objects using LCD fringe projection with phase shifting[J]. Optics Communications, 2001, 189(1-3): 21-29. doi:10.1016/S0030-4018(01)01038-0
[9]	PROLL K P, NIVET J M, KÖRNER K, et al. Microscopic three-dimensional topometry with ferroelectric liquid-crystal-on-silicon displays[J]. Applied Optics, 2003, 42(10): 1773-1778. doi:10.1364/AO.42.001773
[10]	NOTNI G, RIEHEMANN S, KUEHMSTEDT P, et al. OLED microdisplays: a new key element for fringe projection setups[J]. Proceedings of SPIE, 2004, 5532: 170-177. doi:10.1117/12.560433
[11]	ZHANG SH F, LI B, REN F J, et al. High-precision measurement of binocular telecentric vision system with novel calibration and matching methods[J]. IEEE Access, 2019, 7: 54682-54692. doi:10.1109/ACCESS.2019.2913181
[12]	ZHANG CH P, HUANG P S, CHIANG F P. Microscopic phase-shifting profilometry based on digital micromirror device technology[J]. Applied Optics, 2002, 41(28): 5896-5904. doi:10.1364/AO.41.005896
[13]	LIU Y H, ZHANG Q C, ZHANG H H, et al. Improve temporal Fourier transform profilometry for complex dynamic three-dimensional shape measurement[J]. Sensors, 2020, 20(7): 1808. doi:10.3390/s20071808
[14]	ZHANG H H, ZHANG Q C, LI Y, et al. High speed 3D shape measurement with temporal Fourier transform profilometry[J]. Applied Sciences, 2019, 9(19): 4123. doi:10.3390/app9194123
[15]	史耀群, 邓林嘉, 王朝旭, 等. 一种基于结构光条纹投影的微小物体测量系统[J]. 应用光学,2019,40(6):1120-1125. doi:10.5768/JAO201940.0603007 SHI Y Q, DENG L J, WANG ZH X, et al. Micro-objects measurement system based on structured light fringe projection[J]. Journal of Applied Optics, 2019, 40(6): 1120-1125. (in Chinese) doi:10.5768/JAO201940.0603007
[16]	ZHONG M, CUI J, HYUN J S, et al. Uniaxial three-dimensional phase-shifting profilometry using a dual-telecentric structured light system in micro-scale devices[J]. Measurement Science and Technology, 2020, 31(8): 085003. doi:10.1088/1361-6501/ab63b2
[17]	殷永凯, 张宗华, 刘晓利, 等. 条纹投影轮廓术系统模型与标定综述[J]. 红外与工程,2020,49(3):0303008. doi:10.3788/IRLA202049.0303008 YIN Y K, ZHANG Z H, LIU X L, et al. Review of the system model and calibration for fringe projection profilometry[J]. Infrared and Laser Engineering, 2020, 49(3): 0303008. (in Chinese) doi:10.3788/IRLA202049.0303008
[18]	ZHANG ZH Y. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334. doi:10.1109/34.888718
[19]	LI B W, KARPINSKY N, ZHANG S. Novel calibration method for structured-light system with an out-of-focus projector[J]. Applied Optics, 2014, 53(16): 3415-3426. doi:10.1364/AO.53.003415
[20]	LI B W, ZHANG S. Flexible calibration method for microscopic structured light system using telecentric lens[J]. Optics Express, 2015, 23(20): 25795-25803. doi:10.1364/OE.23.025795
[21]	LIU H B, LIN H J, YAO L SH. Calibration method for projector-camera-based telecentric fringe projection profilometry system[J]. Optics Express, 2017, 25(25): 31492-31508. doi:10.1364/OE.25.031492
[22]	安东, 陈李, 丁一飞, 等. 光栅投影相位法系统模型及标定方法[J]. 中国光学,2015,8(2):248-254. doi:10.3788/co.20150802.0248 AN D, CHEN L, DING Y F, et al. Optical system model and calibration of grating projection phase method[J]. Chinese Optics, 2015, 8(2): 248-254. (in Chinese) doi:10.3788/co.20150802.0248
[23]	丁一飞, 王永红, 胡悦, 等. 样本块匹配光栅投影阶梯标定方法[J]. 中国测试,2016,42(8):7-12. doi:10.11857/j.issn.1674-5124.2016.08.002 DING Y F, WANG Y H, HU Y, et al. Step calibration method of grating projection based on exemplar matching[J]. China Measurement& Test, 2016, 42(8): 7-12. (in Chinese) doi:10.11857/j.issn.1674-5124.2016.08.002
[24]	LU P, SUN CH K, LIU B, et al. Accurate and robust calibration method based on pattern geometric constraints for fringe projection profilometry[J]. Applied Optics, 2017, 56(4): 784-794. doi:10.1364/AO.56.000784
[25]	CHEN Z, LIAO H Y, ZHANG X M. Telecentric stereo micro-vision system: calibration method and experiments[J]. Optics and Lasers in Engineering, 2014, 57: 82-92. doi:10.1016/j.optlaseng.2014.01.021
[26]	HU Y, CHEN Q, LI H Y, et al. Absolute three-dimensional micro surface profile measurement based on a Greenough-type stereomicroscope[J]. Measurement Science and Technology, 2017, 28(4): 045004. doi:10.1088/1361-6501/aa5a2d
[27]	Overview: DLP products[EB/OL]. [2020-10-18]. http://www.ti.com/dlp-chip/overview.html.
[28]	肖萍萍. 基于光栅投射的小尺寸物体三维形状测量系统研究[D]. 武汉: 华中科技大学, 2019. XIAO P P. Research on 3D shape measurement system of small scale object based on grating projection[D]. Wuhan: Huazhong University of Science and Technology, 2019. (in Chinese).
[29]	VAN DER JEUGHT S, SOONS J A M, DIRCKX J J J. Real-time microscopic phase-shifting profilometry[J]. Applied Optics, 2015, 54(15): 4953-4959.
[30]	CHEN L C, LIAO CH CH, LAI M J. Full-field micro surface profilometry using digital fringe projection with spatial encoding principle[J]. Journal of Physics: Conference Series, 2005, 13: 147-150. doi:10.1088/1742-6596/13/1/034
[31]	LI A M, PENG X, YIN Y K, et al. Fringe projection based quantitative 3D microscopy[J]. Optik, 2013, 124(21): 5052-5056. doi:10.1016/j.ijleo.2013.03.070
[32]	LI D, LIU CH Y, TIAN J D. Telecentric 3D profilometry based on phase-shifting fringe projection[J]. Optics Express, 2014, 22(26): 31826-31835. doi:10.1364/OE.22.031826
[33]	PENG J ZH, WANG M, DENG N N, et al. Distortion correction for microscopic fringe projection system with Scheimpflug telecentric lens[J]. Applied Optics, 2015, 54(34): 10055-10062. doi:10.1364/AO.54.010055
[34]	YIN Y K, WANG M, GAO B Z, et al. Fringe projection 3D microscopy with the general imaging model[J]. Optics Express, 2015, 23(5): 6846-6857. doi:10.1364/OE.23.006846
[35]	WANG M, YIN Y K, DENG D N, et al. Improved performance of multi-view fringe projection 3D microscopy[J]. Optics Express, 2017, 25(16): 19408-19421. doi:10.1364/OE.25.019408
[36]	HU Y, CHEN Q, FENG SH J, et al. A new microscopic telecentric stereo vision system-calibration, rectification, and three-dimensional reconstruction[J]. Optics and Lasers in Engineering, 2019, 113: 14-22. doi:10.1016/j.optlaseng.2018.09.011
[37]	HU Y, LIANG Y CH, TAO T Y, et al. Dynamic 3D measurement of thermal deformation based on geometric-constrained stereo-matching with a stereo microscopic system[J]. Measurement Science and Technology, 2019, 30(12): 125007. doi:10.1088/1361-6501/ab35a1
[38]	QUAN C, TAY C J, HE X Y, et al. Microscopic surface contouring by fringe projection method[J]. Optics& Laser Technology, 2002, 34(7): 547-552.
[39]	WANG W H, WONG Y S, HONG G S. 3D measurement of crater wear by phase shifting method[J]. Wear, 2006, 261(2): 164-171. doi:10.1016/j.wear.2005.09.036
[40]	张莲涛, 卢荣胜, 程子怡. 基于相移偏折法的高反射表面面形测量技术[J]. 光子学报,2020,49(1):0112002. doi:10.3788/gzxb20204901.0112002 ZHANG L T, LU R SH, CHENG Z Y. Measurement technique of high-reflected surface based on phase measuring deflectometry[J]. Acta Photonica Sinica, 2020, 49(1): 0112002. (in Chinese) doi:10.3788/gzxb20204901.0112002
[41]	LIU X H, ZHANG Z H, GAO N, et al. 3D shape measurement of diffused/specular surface by combining fringe projection and direct phase measuring deflectometry[J]. Optics Express, 2020, 28(19): 27561-27574. doi:10.1364/OE.402432
[42]	陶迁, 周志峰, 吴明晖, 等. 基于相位测量偏折术的反射表面缺陷检测[J]. 液晶与显示,2020,35(12):1315-1322. doi:10.37188/YJYXS20203512.1315 TAO Q, ZHOU ZH F, WU M H, et al. Detection of reflective surface defects based on phase measuring deflectometry[J]. Chinese Journal of Liquid Crystals and Displays, 2020, 35(12): 1315-1322. (in Chinese) doi:10.37188/YJYXS20203512.1315
[43]	王月敏, 张宗华, 高楠. 基于全场条纹反射的镜面物体三维面形测量综述[J]. 光学精密工程,2018,26(5):1014-1027. doi:10.3788/OPE.20182605.1014 WANG Y M, ZHANG Z H, GAO N. Review on three-dimensional surface measurements of specular objects based on full-field fringe reflection[J]. Optics and Precision Engineering, 2018, 26(5): 1014-1027. (in Chinese) doi:10.3788/OPE.20182605.1014
[44]	ZHANG L, CHEN Q, ZUO CH, et al. High-speed high dynamic range 3D shape measurement based on deep learning[J]. Optics and Lasers in Engineering, 2020, 134: 106245. doi:10.1016/j.optlaseng.2020.106245
[45]	JIANG H ZH, ZHAO H J, LI X D. High dynamic range fringe acquisition: a novel 3-D scanning technique for high-reflective surfaces[J]. Optics and Lasers in Engineering, 2012, 50(10): 1484-1493. doi:10.1016/j.optlaseng.2011.11.021
[46]	RAO L, DA F P. High dynamic range 3D shape determination based on automatic exposure selection[J]. Journal of Visual Communication and Image Representation, 2018, 50: 217-226. doi:10.1016/j.jvcir.2017.12.003
[47]	ZHANG S. Rapid and automatic optimal exposure control for digital fringe projection technique[J]. Optics and Lasers in Engineering, 2020, 128: 106029. doi:10.1016/j.optlaseng.2020.106029
[48]	CHEN CH, GAO N, WANG X J, et al. Adaptive projection intensity adjustment for avoiding saturation in three-dimensional shape measurement[J]. Optics Communications, 2018, 410: 694-702. doi:10.1016/j.optcom.2017.11.009
[49]	WANG J H, YANG Y X. High-speed three-dimensional measurement technique for object surface with a large range of reflectivity variations[J]. Applied Optics, 2018, 57(30): 9172-9182. doi:10.1364/AO.57.009172
[50]	SONG ZH, JIANG H L, LIN H B, et al. A high dynamic range structured light means for the 3D measurement of specular surface[J]. Optics and Lasers in Engineering, 2017, 95: 8-16. doi:10.1016/j.optlaseng.2017.03.008
[51]	LIU Y ZH, FU Y J, CAI X Q, et al. A novel high dynamic range 3D measurement method based on adaptive fringe projection technique[J]. Optics and Lasers in Engineering, 2020, 128: 106004. doi:10.1016/j.optlaseng.2020.106004
[52]	万钇良, 王建立, 张楠. 一种基于相位相关与子图像的偏振图像配准方法[J]. 液晶与显示,2019,34(5):530-536. doi:10.3788/YJYXS20193405.0530 WAN Y L, WANG J L, ZHANG N. Polarized image registration method based on phase correlation and sub-graph[J]. Chinese Journal of Liquid Crystals and Displays, 2019, 34(5): 530-536. (in Chinese) doi:10.3788/YJYXS20193405.0530
[53]	RIVIERE J, RESHETOUSKI I, FILIPI L, et al. Polarization imaging reflectometry in the wild[J]. ACM Transactions on Graphics, 2017, 36(6): 206.
[54]	FENG SH J, ZHANG Y ZH, CHEN Q, et al. General solution for high dynamic range three-dimensional shape measurement using the fringe projection technique[J]. Optics and Lasers in Engineering, 2014, 59: 56-71. doi:10.1016/j.optlaseng.2014.03.003
[55]	BENVENISTE R, ÜNSALAN C. Binary and ternary coded structured light 3D scanner for shiny objects[M]//GELENBE E, LENT R, SAKELLARI G, et al. .Computer and Information Sciences. Dordrecht: Springer, 2011: 241-244.
[56]	BENVENISTE R, ÜNSALAN C. A color invariant for line stripe-based range scanners[J]. The Computer Journal, 2011, 54(5): 738-753. doi:10.1093/comjnl/bxq014
[57]	BENVENISTE R, ÜNSALAN C. Nary coded structured light-based range scanners using color invariants[J]. Journal of Real-Time Image Processing, 2014, 9(2): 359-377. doi:10.1007/s11554-011-0235-4
[58]	MENG L F, LU L Y, BEDARD N, et al. . Single-shot specular surface reconstruction with gonio-plenoptic imaging[C]. Proceedings of 2015 IEEE International Conference on Computer Vision, IEEE, 2015.
[59]	ZHANG L, CHEN Q, ZUO CH, et al. High dynamic range and real-time 3D measurement based on a multi-view system[J]. Proceedings of SPIE, 2019, 11427: 1142715.
[60]	HU Y, CHEN Q, LIANG Y CH, et al. Microscopic 3D measurement of shiny surfaces based on a multi-frequency phase-shifting scheme[J]. Optics and Lasers in Engineering, 2019, 122: 1-7. doi:10.1016/j.optlaseng.2019.05.019