-
摘要:折反射周视系统作为近十几年发展起来的一种新型周视视觉实现形式,相比相机旋转扫描、多相机图像拼接和鱼眼镜头大视场成像等常规方法,在小型化、结构灵活性、成本和实时性方面具有优势。本文综述了折反射周视系统的成像模型、系统标定、畸变校正和全视场清晰成像等基本问题研究状况,讨论了折反射周视系统在红外成像和立体视觉领域的扩展应用研究现状,最后总结了目前存在的问题,并提出未来折反射周视成像系统将围绕非单视点成像模型、提高空间分辨力的方法和处理算法实时实现开展研究。Abstract:As a new panoramic vision system developed in recent 10 years, the catadioptric panoramic system has advantages in miniaturization, structural flexibility, low cost and real-time compared to conventional methods such as camera rotation scanning, multi-camera image stitching, and fisheye lens large field imaging. In this review, the recent progress in basic catadioptric panoramic system, including imaging models, system calibration, distortion correction and overall well-focused imaging, are summarized. Moreover, the extended applications in infrared imaging and stereo vision are also presented. Finally, the existing unsolved problems are discussed. It is also suggested that the future catadioptric panoramic system will focus on non-single view imaging model, improving of the spatial resolution method and implementing of the real-time processing algorithm.
-
表 13个系统参数对比
Table 1.Comparison of three system′s parameters
IRC 360 STRIX360 Lockheed Martin系统 光谱范围 长波 3.25~5.1 μm 3.4~4.9 μm 视场角范围 方位360°,俯仰-58°~10° 方位360°, 俯仰-10°~30° 方位360°, 俯仰-10°~50° 探测器 768×1 024, VOx,
17 μm ×17 μm2 040×2 040, InSb,
15 μm ×15 μm640×512, InSb,
20 μm×20 μm帧率/Hz 30 30 120 -
[1] US Army. Panoramic television viewing system:America, US, 3505465 A[P].1970-04-07. [2] YAMAZAWA K, YAGI Y, YACHIDA M. Omnidirectional imaging with hyperboloidal projection[C]. 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan, 1993:1029-1034. [3] PEGARD C, MOUADDIB E M. A mobile robot using a panorama view[C]. IEEE International Conference on Robotics and Automation, IEEE, 1996:89-94. [4] NAYAR S K. Catadioptric omnidirectional camera[C]. IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 1997:482-488. [5] CHAHL J S, SRINIVASAN M V. Reflective surfaces for panoramic imaging[J]. Applied Optics, 1997, 36(31):8275-8285.doi:10.1364/AO.36.008275 [6] CONROY T, MOORE J T. Resolution invariant surfaces for panoramic vision systems[C]. 7th IEEE International Conference on Computer Vision, IEEE, 1999:392. [7] HICKS R A, BAJCSY R. Reflective surfaces as computational sensors[J]. Image and Vision Computing, 1999, 19(11):773-777. [8] GACHTER S. Mirror design for an omnidirectional camera with a uniform cylindrical projection when using the SVAVISCA sensor[R]. Research reports of CMP, OMNIVIEWS Project, Czech Technical University in Prague, 2001. [9] BAKER S, NAYAR S K. A theory of catadioptric image formation[C]. 6th IEEE International Conference on Computer Vision, IEEE, 1998:35-42. [10] BAKER S, NAYAR S K. A theory of single-viewpoint catadioptric image formation[J]. Int. J. Comput. Vis., 1999, 35(2):175-196.doi:10.1023/A:1008128724364 [11] 贺宇, 王岭雪, 周星光, 等.单视点红外折反射全向成像系统参数设计[J].光学学报, 2016, 36(1):0111002.http://youxian.cnki.com.cn/yxdetail.aspx?filename=ZGGA201710014&dbname=CJFDPREPHE Y, WANG L X, ZHOU X G, et al.. Parameters design for single viewpoint infrared omnidirectional view sensors[J]. Acta Optica Sinica, 2016, 36(1):0111002.(in Chinese)http://youxian.cnki.com.cn/yxdetail.aspx?filename=ZGGA201710014&dbname=CJFDPREP [12] NAGAHARA H, YOSHIDA K, YACHIDA M. An omnidirectional vision sensor with single view and constant resolution[C]. IEEE International Conference on Computer Vision, IEEE, 2007:1-8. [13] HICKS R A, MILLSTONE M, DANⅡLIDⅡS K. Realizing any central projection with a mirror pair[J]. Applied Optics, 2006, 45(28):7205-7210.doi:10.1364/AO.45.007205 [14] STURZL W, SRINIVASAN M V. Omnidirectional imaging system with constant elevational gain and single viewpoint[C]. 10th Workshop on Omnidirectional Vision, CameraNetworks and Non-classical Cameras, Zaragoa, Spain, 2010:1-7. [15] SWAMINATHAN R, GROSSBERG M D, NAYAR S K. Non-single viewpoint catadioptric cameras:geometry and analysis[J]. Int. J. Comput. Vis., 2006, 66(3):211-229.doi:10.1007/s11263-005-3220-1 [16] SWAMINATHAN R, GROSSBERG M D, NAYAR S K. Caustics of catadioptric cameras[C]. IEEE International Conference on Computer Vision, IEEE, 2001:2-9. [17] SWAMINATHAN R, GROSSBERG M D, NAYAR S K. A perspective on distortions[C]. IEEE Computer Vision and Pattern Recognition, IEEE, 2003:594-601. [18] YU J Y, MCMILLAN L. Modelling reflections via multiperspective imaging[J]. IEEE Computer Vision and Pattern Recognition, 2005, 1:117-124. [19] YU J Y, MCMILLAN L. General linear cameras[C]. 8th European. Conf. on Computer vision(ECCV 2004), Prague, Czech, 2004:14-27. [20] GEYER C, DANⅡLIDIS K. A unifying theory for central panoramic systems and practical applications[C]. 6th European. Conf. on Computer vision(ECCV 2000), Antibes, France, 2000:445-461. [21] GEYER C, DANⅡLIDIS K. Mirrors in motion:Epipolar geometry and motion estimation[J]. Int. J. Comput. Vis., 2003, 2:766-773. [22] GEYER C M. Catadioptric projective geometry:theory and applications[D]. Philadelphia:University of Pennsylvania, 2003. [23] ALIAGA D G. Accurate catadioptric calibration for real-time pose estimation in room-size environments[C]. 8th IEEE International Conference on Computer Vision, IEEE, 2001:127-134. [24] PUIG L, BASTANLAR Y, STURM P, et al.. Calibration of central catadioptric cameras using a dlt-like approach[J]. Int. J. Comput. Vis., 2011, 93(1):101-114.doi:10.1007/s11263-010-0411-1 [25] THIRTHALA S R, POLLEFEYS M. Radial multi-focal tensors[J]. Int. J. Comput. Vis., 2012, 96(2):195-211.doi:10.1007/s11263-011-0463-x [26] WU Y H, HU Z Y. Geometric invariants and applications under catadioptric camera model[C]. IEEE International Conference on Computer Vision, IEEE, 2005:1547-1554. [27] GEYER C, DANⅡLIDIS K. Paracatadioptric camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(5):687-695.doi:10.1109/34.1000241 [28] VANDEPORTAELE B, CATTOEN M, MARTHON P, et al. A new linear calibration method for paracatadioptric cameras[C]. 18th IEEE International Conference on Pattern Recognition, IEEE, 2006:647-651. [29] VASSEUR P, MOUADDIB E M. Central catadioptric line detection[C]. In British Machine Vision Conference, London, UK, 2004:doi:10.52441C,18.8. [30] CAGLIOTI V, TADDEI P, BORACCH I, et al.. Singleimage calibration of off-axis catadioptric cameras using lines[C]. 11th IEEE International Conference on Computer Vision, IEEE, 2007:1-6. [31] YING X H, HU Z Y. Catadioptric camera calibration using geometric invariants[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(10):1260-1271.doi:10.1109/TPAMI.2004.79 [32] WU Y H, LI Y F, HU Z Y. Easy calibration for para-catadioptric-like camera[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, 2006:5719-5724. [33] YING X H, ZHA H B. Simultaneously calibrating catadioptric camera and detecting line features using hough transform[C]. IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, 2005:412-417. [34] YING X H, ZHA H B. Identical projective geometric properties of central catadioptric line images and sphere images with applications to calibration[J]. Int. J. Comput. Vis., 2008, 78(1):89-105.doi:10.1007/s11263-007-0082-8 [35] WU F C, DUAN F Q, HU Z Y, et al. A new linear algorithm for calibrating central catadioptric cameras[J]. Pattern Recognition, 2008, 41(10):3166-3172.doi:10.1016/j.patcog.2008.03.010 [36] SCARAMUZZA D, MARTINELLI A, SIEGWART R. A flexible technique for accurate omnidirectional camera calibration and structure from motion[C]. IEEE International Conference on Computer Vision Systems, IEEE, 2006:45. [37] MEI C, RIVES P. Single view point omnidirectional camera calibration from planar grids[C]. IEEE International Conference on Robotics and Automation, IEEE, 2007:3945-3950. [38] GASPARINI S, STURM P, BARRETO J P. Plane-based calibration of central catadioptric cameras[C]. 12th IEEE International Conference on Computer Vision, IEEE, 2009:1195-1202. [39] DENG X M, WU F C, WU Y H. An easy calibration method for central catadioptric cameras[J]. Acta Automatica Sinica, 2007, 33(8):801-808.doi:10.1360/aas-007-0801 [40] KANG S B. Catadioptric self-calibration[C]. IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2000:201-207. [41] RAMALINGAM S, STURM P, LODHA S K. Generic self-calibration of central cameras[J]. Computer Vision and Image Understanding, 2010, 114(2):210-219.doi:10.1016/j.cviu.2009.07.007 [42] ESPUNY F, BURGOS GIL J I. Generic self-calibration of central cameras from two rotational ows[J]. Int. J. Comput. Vis., 2011, 91(2):131-145.doi:10.1007/s11263-010-0335-9 [43] GASPAR J, SANTOS-VICTOR J. Visual path following with a catadioptric panoramic camera[C]. International Symposium on Intelligent Robotic Systems, IEEE, 1999:139-147. [44] J.MAYBANK S, IENG S, BENOSMAN R. A fisher-rao metric for paracatadioptric images of lines[J]. Int. J. Comput. Vis., 2012, 99(2):147-165.doi:10.1007/s11263-012-0523-x [45] TANG Y Z, LI Y F, LUO J. Parametric distortion-adaptive neighborhood for omnidirectional camera[J]. Applied Optics, 2015, 54(23):6969-6978.doi:10.1364/AO.54.006969 [46] 曾吉勇, 苏显渝.水平场景无畸变折反射周视成像系统透镜畸变的消除[J].光学学报, 2004, 24(6):730-734.http://youxian.cnki.com.cn/yxdetail.aspx?filename=ZGGA201710014&dbname=CJFDPREPZENG J Y, SU X Y. Elimination of the lens distortion in catadioptric omnidirectional distortionless imaging system for horizontal scene[J]. Acta Optica Sinica, 2004, 24(6):730-734.(in Chinese)http://youxian.cnki.com.cn/yxdetail.aspx?filename=ZGGA201710014&dbname=CJFDPREP [47] 肖潇, 杨国光, 白剑.基于球面透视投影约束的周视环形透镜畸变校正[J].光学学报, 2008, 28(4):675-680.http://youxian.cnki.com.cn/yxdetail.aspx?filename=ZGGA201710014&dbname=CJFDPREPXIAO X, YANG G G, BAI J. Panoramic-Annular-Lens distortion correction based on spherical perspective projection constraint[J]. Acta Optica Sinica, 2008, 28(4):675-680.(in Chinese)http://youxian.cnki.com.cn/yxdetail.aspx?filename=ZGGA201710014&dbname=CJFDPREP [48] WU Y H, HU Z Y, LI Y F. Radial distortion invariants and lens evaluation under a single-optical-axis omnidirectional camera[J]. Computer Vision and Image Understanding, 2014, 126(2):11-27.http://dl.acm.org/citation.cfm?id=3030862 [49] SWAMINATHAN R. Focus in catadioptric imaging systems[C]. IEEE 11th International Conference on Computer Vision, IEEE, 2007:1-7. [50] LI W M, LI Y F. An analytical solution to optimal focal distance in catadioptric imaging systems[C]. IEEE International Conference on Robotics and Automation, IEEE, 2011:6300-6305. [51] LI W M, LI Y F. Overall well-focused catadioptric image acquisition with multifocal images:a model-based method[J]. IEEE Transactions on Image Processing, 2012, 21(8):3697-3706.doi:10.1109/TIP.2012.2195010 [52] 李永乐, 张茂军, 娄静涛, 等.去散焦模糊的折反射全向成像系统设计[J].光学学报, 2012, 32(9):0911001.http://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201209013.htmLI Y L, ZHANG M J, LOU J T, et al.. Design of catadioptric omnidirectional imaging system for defocus deblurring[J]. Acta Optica Sinica, 2012, 32(9):0911001.(in Chinese)http://www.cnki.com.cn/Article/CJFDTOTAL-GXXB201209013.htm [53] LIU Y, LI H F, LI Y L, et al. Coded aperture enhanced catadioptric optical system for omnidirectional image deblurring[J]. Optik, 2014, 125(1):11-17.doi:10.1016/j.ijleo.2013.05.146 [54] LI Y L, LOU J T, XU Y C. Defocus deblurring for catadioptric omnidirectional imaging based on coded aperture and omni-total variation[J]. Advanced Robotics, 2015, 29(16):1-13. [55] LIU Y, LI Y L, LOU J T, et al.. Omni-total variation algorithm for the restoration of all-focused catadioptric image[J]. Optik, 2014, 125(14):3685-3689.doi:10.1016/j.ijleo.2014.01.068 [56] 白瑜, 廖志远, 廖胜, 等.共孔径消热差红外双波段光学系统[J].光学精密工程, 2016, 24(2):268-277.http://cpfd.cnki.com.cn/Article/CPFDTOTAL-ZXGH201507003068.htmBAI Y, LIAO Z Y, LIAO SH, et al.. Infrared dual band a thermal optical system with common aperture[J]. Opt. Precision Eng., 2016, 24(2):268-277.(in Chinese)http://cpfd.cnki.com.cn/Article/CPFDTOTAL-ZXGH201507003068.htm [57] WILLETT R M, MARCIA R F, NICHOLS J M. Compressed sensing for practical optical imaging systems:a tutorial[J]. Optical Engineering, 2011, 50(7):072601.doi:10.1117/1.3596602 [58] ABURMAD S. Panoramic thermal imaging:challenges and tradeoffs[J]. SPIE, 2014, 9070:90700W.http://adsabs.harvard.edu/abs/2014SPIE.9070E..0EA [59] NICHOLS J M, WATERMAN J R, BAYYA S, et al.. Influence of spinel head window thickness on the performance characteristics of a submarine, panoramic, infrared imaging system[J]. SPIE, 2011, 8012:90122C.doi:10.1117/12.884521.full [60] NICHOLS J M, WATERMAN J R, MENON R, et al.. Modeling and analysis of a high-performance midwave infrared panoramic periscope[J]. Optical Engineering, 2010, 49(11):113202.doi:10.1117/1.3505866 [61] FURXHI O, DRIGGERS R G, HOLST G, et al.. Performance analysis of panoramic infrared systems[J]. SPIE, 2014, 9071:907112.doi:10.1117/12.2067671.full [62] TANG Y Z, LI Y F. Contour coding based rotating adaptive model for human detection and tracking in thermal catadioptric omnidirectional vision[J]. Applied Optics, 2012, 51(27):6641-6652.doi:10.1364/AO.51.006641 [63] TANG Y Z, LI Y F, BAI T X, et al.. Rotating adaptive Haar wavelet transform for human tracking in thermal omnidirectional vision[C]. IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, IEEE, 2012:324-329. [64] TANG Y Z, LUO J, LI Y F, et al.. Rotational kinematics model based adaptive particle filter for robust human tracking in thermal omnidirectional vision[J]. Mathematical Problems in Engineering, 2015:347497. [65] TANG Y Z, LI Y F, BAI T X, et al.. A rotating adaptive model for human tracking in thermal catadioptric omnidirectional vision[C]. IEEE International Conference on Robotics and Biomimetics, IEEE, 2011:2435-2440. [66] RemoteReality. Strix360 camera system[EB/OL].[2015-01-11].http://www.remotereality.com. [67] BJORK C, WAN W. Mid-wave infrared(MWIR) panoramic sensor for various applications[J]. SPIE, 2010, 7660:76600B. [68] 周星光, 贺宇, 王岭雪, 等.单视点双曲面折反射红外全景成像系统设计与分析[J].红外与 工程, 2016, 45(9):197-205.http://youxian.cnki.com.cn/yxdetail.aspx?filename=ZGGA201710014&dbname=CJFDPREPZHOU X G, HE Y, WANG L X, et al.. Hyperbolicsingle-viewpoint infrared catadioptric panoramic imaging system[J]. Infrared and Laser Engineering, 2016, 45(9):197-205.(in Chinese)http://youxian.cnki.com.cn/yxdetail.aspx?filename=ZGGA201710014&dbname=CJFDPREP [69] 贺宇, 王岭雪, 蔡毅, 等.恒等探测距离折反射周视红外成像系统设计及分析[J].光学学报, 2017, 37(4):0422002.http://youxian.cnki.com.cn/yxdetail.aspx?filename=ZGGA201710014&dbname=CJFDPREPHE Y, WANG L X, CAI Y, et al.. Design and analysis for catadioptric omnidirectional view Infrared imaging system with constant detection range[J]. Acta Optica Sinica, 2017, 37(4):0422002.(in Chinese)http://youxian.cnki.com.cn/yxdetail.aspx?filename=ZGGA201710014&dbname=CJFDPREP [70] SCHARSTEIN D, SZELISKI R.http://vision.middlebury.edu/stereo/eval/[EB/OL].[2016-07-06]. [71] SCHARSTEIN D, SZELISKI R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J]. Int. J. Comput. Vis., 2002, 47(1):7-42.doi:10.1023/A:1014573219977 [72] HIRSCHMULLER H. Stereo vision in structured environments by consistent semi-global matching[C]. 21rd IEEE Conference on Computer Vision and Pattern Recognition(CVPR), IEEE, 2006:2386-2393. [73] TOMBARI F, MATTOCCIA S, STEFANO L D, et al.. Near real-time stereo based on effective cost aggregation[C]. 22rd IEEE Conference on Computer Vision and Pattern Recognition(CVPR), IEEE, 2008:1-4. [74] MATTOCCIA S, GIARDINO S, GAMBINI A. Accurate and efficient cost aggregation strategy for stereo correspondence based on approximated joint bilateral filtering[C]. Asian Conference on Computer Vision, Queenstown, New Zealand, 2010, 5995:371-380. [75] MATTOCCIA S. A locally global approach to stereo correspondence[C]. IEEE International Conference on Computer Vision Workshops, IEEE, 2009:1763-1770. [76] MATTOCCIA S. Improving the accuracy of fast dense stereo correspondence algorithms by enforcing local consistency of disparity fields[C]. 3D Data Processing, Visualization, and Transmission, Paris, Frame, 2010:17-20. [77] MATTOCCIA S. Fast locally consistent dense stereo on multicore[C]. 6th IEEE Embedded Computer Vision Workshop(ECVW2010), CVPR workshop, IEEE, 2010:69-76. [78] MATTOCCIA S. Accurate dense stereo by constraining local consistency on superpixels[C]. 20th International Conference on Pattern Recognition, Istambul, Turkey, 2010:1832-1835. [79] ZHANG C, LI Z W, CHENG Y H, et al.. MeshStereo:a global stereo model with mesh alignment regularization for view interpolation[C]. 15th IEEE International Conference on Computer Vision(ICCV), IEEE, 2015:2057-2065. [80] 宋涛, 熊文莉, 侯培国, 等.基于极曲线几何和支持邻域的鱼眼图像立体匹配[J].光学精密工程, 2016, 24(8):2050-2058.http://www.cnki.com.cn/Article/CJFDTOTAL-GXJM201608028.htmSONG T, XIONG W L, HOU P G, et al.. Stereo matching for fish-eye images based on epipolar geometry and support neighborhood[J]. Opt. Precision Eng., 2016, 24(8):2050-2058.(in Chinese)http://www.cnki.com.cn/Article/CJFDTOTAL-GXJM201608028.htm [81] 王晓燕, 王世刚, 姜秀红, 等.亮度优化立体视频视觉舒适度评价[J].中国光学, 2015, 8(3):394-400.//www.illord.com/CN/abstract/abstract9300.shtmlWANG X Y, WANG SH G, JIANG X H, et al. Evaluation of stereo video visual comfort based on luminance optimization[J]. Chinese Optics, 2015, 8(3):394-400.(in Chinese)//www.illord.com/CN/abstract/abstract9300.shtml [82] 高礼圳, 刘书桂, 韩振华.零件的角点提取及匹配定位[J].中国光学, 2016, 9(4):397-404.//www.illord.com/CN/abstract/abstract9445.shtmlGAO L ZH, LIU SH G, HAN ZH H. Corner extraction and matching location of parts[J]. Chinese Optics, 2016, 9(4):397-404.(in Chinese)//www.illord.com/CN/abstract/abstract9445.shtml [83] GLUCKMAN J, NAYAR S K, THORESZ K J. Real-time omnidirectional and panoramic stereo[C]. 1998 DARPA Image Understanding Workshop, California, USA, 1998:299-303. [84] KOYASU H S, MIURA J, SHIRAI Y. Recognizing moving obstacles for robot navigation using real-time omnidirectional stereo vision[J]. J. Robotics and Mechatronics, 2002, 14(2):147-156.doi:10.20965/jrm.2002.p0147 [85] CABRAL E L L, SOUZA J J C, HUNOLD M C. Omnidirectional stereo vision with a hyperbolic double lobed mirror[C]. 17th International Conference on Pattern Recognition, IEEE, 2004:1-9. [86] JANG G, KIM S, KWEON I. Single-camera panoramic stereo system with single-viewpoint optics[J]. Optics Letters, 2006, 31(1):41-43.doi:10.1364/OL.31.000041 [87] SU L C, ZHU F. Design of a novel omnidirectional stereo vision system[J]. Acta Automatica Sinica, 2006, 32(1):67-72. [88] LI W M, LI Y F. Single-camera panoramic stereo imaging system with a fisheye lens and aconvex mirror[J]. Optics Express, 2011, 19(7):5855-5867.doi:10.1364/OE.19.005855 [89] HUANG Z, BAI J, HOU X Y. Design of panoramic stereo imaging with single optical system[J]. Optics Express, 2012, 20(6):6085-6096.doi:10.1364/OE.20.006085 [90] 黄治. 单传感器全景立体环带成像光学系统的研究[D]. 杭州: 浙江大学, 2014.http://cdmd.cnki.com.cn/Article/CDMD-10335-1014269148.htmHUANG ZH. Study of panoramic stereo imaging with single optical system[D]. Hangzhou:Zhejiang University, 2014.(in Chinese)http://cdmd.cnki.com.cn/Article/CDMD-10335-1014269148.htm [91] HE Y, WANG L X, CAI Y, et al.. Monocular catadioptric panoramic depth estimation via caustics-based virtual scene transition[J]. J. Optical Society of America A, 2016, 33(9):1872-1879.doi:10.1364/JOSAA.33.001872 [92] CARUSO D, ENGEL J, CREMERS D. Large-scale direct SLAM for omnidirectional cameras[C]. IEEE International Conference on Intelligent Robort & Systems, IEEE, 2015:141-148.