留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

全景内窥成像技术及应用

霍嘉燚,  李冕豪,  王子川,  袁波,  杨青,  王立强

文章导航 >   > 2023 >  16(1): 44-60.
霍嘉燚, 李冕豪, 王子川, 袁波, 杨青, 王立强. 全景内窥成像技术及应用[J]. , 2023, 16(1): 44-60. doi: 10.37188/CO.2022-0074
引用本文: 霍嘉燚, 李冕豪, 王子川, 袁波, 杨青, 王立强. 全景内窥成像技术及应用[J]. , 2023, 16(1): 44-60. doi: 10.37188/CO.2022-0074
shu
HUO Jia-yi, LI Mian-hao, WANG Zi-chuan, YUAN Bo, YANG Qing, WANG Li-qiang. Panoramic endoscopic imaging technology and it’s applications[J]. Chinese Optics, 2023, 16(1): 44-60. doi: 10.37188/CO.2022-0074
Citation: HUO Jia-yi, LI Mian-hao, WANG Zi-chuan, YUAN Bo, YANG Qing, WANG Li-qiang. Panoramic endoscopic imaging technology and it’s applications[J]. Chinese Optics, 2023, 16(1): 44-60. doi: 10.37188/CO.2022-0074
shu

全景内窥成像技术及应用

  • 1.

    浙江大学 光电科学与工程学院, 浙江 杭州 310027

  • 2.

    之江实验室 类人感知研究中心, 浙江 杭州 311100

基金项目: 国家重点研发计划项目（No. 2021YFC2400103）；浙江省基础公益研究计划项目（No. LGF20F50006）；之江实验室科研项目（No. 2019MC0AD02, 2022MG0AL01）
详细信息
    作者简介:

    霍嘉燚（1997—），男，河北保定人，博士研究生，2019年6月于天津大学获得学士学位，2019年9月进入浙江大学光电学院学习，主要从事内窥镜成像、三维成像等方面研究。E-mail：jiayi_huo@zju.edu.cn

    王立强（1977—），男，陕西渭南人，副教授，博士生导师，1998年、2001年、2004年于浙江大学分别获得学士学位，硕士学位，博士学位，现为浙江大学光电信息检测技术研究所副所长，主要从事光电成像技术及内窥镜方面的研究。E-mail：wangliqiang@zju.edu.cn

  • 中图分类号: TP394.1

Panoramic endoscopic imaging technology and it’s applications

  • 1.

    College of Optical Science and Engineering, Zhejiang University, Hangzhou 310027, china

  • 2.

    Research Center for Humanoid Sensing, Zhejiang Lab., Hangzhou 311100, china

Funds: Supported by the National Key Research and Development Program of China (No. 2021YFC2400103); the Zhejiang Provincial Natural Science Foundation of China (No. LGF20F050006); Key Research Project of Zhejiang Lab (No. 2019MC0AD02, No. 2022MG0AL01)
More Information

    摘要
  • 摘要:

    全景内窥成像技术可有效减小体内器官的观察盲区，具有缩短手术时间、降低术中出血风险、改善手术预后、缩短术后恢复时间等多种优点，在微创手术和术前检查中有重要应用价值，是近年来的研究热点。本文从原理和产品应用两个方面对全景内窥成像技术进行了梳理。首先，综述了基于二维和三维成像的各种全景内窥成像技术，阐述了它们各自的实现方式，并分析了其关键指标和性能。其次，对比分析了由全景内窥成像技术衍生出来的胶囊内窥镜、全景结直肠镜等多种不同类型的产品，并展望了全景内窥成像技术的发展趋势和应用前景。

     

    Abstract:

    Panoramic endoscopic imaging technology can effectively reduce the observation blind area of internal organs. It has many advantages, such as shortening the operation time, reducing the risk of intraoperative bleeding, improving the prognosis and shortening the postoperative recovery time. It has important application value in minimally invasive surgery and preoperative examination. It is a research hotspot in recent years. This paper combs the panoramic endoscopic imaging technology from two aspects: principle and product applications. Firstly, various panoramic endoscopic imaging technologies based on two-dimensional and three-dimensional imaging are reviewed, their implementation methods are described, and their key indexes and performances are analyzed. Secondly, the capsule endoscope, panoramic enteroscope and other different types of products derived from panoramic endoscopic imaging technology are compared and analyzed, and the development trend and application prospect of panoramic endoscopic imaging technology are prospected.

     

    • HTML全文

  • 图 1  套针相机组件[26]

    Figure 1.  Trocar-camera assembly[26]

    下载: 全尺寸图片 幻灯片

    图 2  双摄像头胶囊内窥镜[30]

    Figure 2.  Dual-view capsule endoscope[30]

    下载: 全尺寸图片 幻灯片

    图 3  紧凑双视角内窥镜设计方案[33]

    Figure 3.  Design scheme of compact dual-view endoscope[33]

    下载: 全尺寸图片 幻灯片

    图 4  自由曲面棱镜的设计方案[33]

    Figure 4.  Design scheme of freeform surface lens[33]

    下载: 全尺寸图片 幻灯片

    图 5  全景环形透镜的原理示意图

    Figure 5.  Principle schematic diagram of panoramic annular lens

    下载: 全尺寸图片 幻灯片

    图 6  基于全景环形透镜和自由曲面棱镜的内窥镜系统[34]

    Figure 6.  Endoscope system based on PAL and freeform surface lens[34]

    下载: 全尺寸图片 幻灯片

    图 7  凹凸型广角内窥镜设计方案[36]

    Figure 7.  Design of foveated wide angle endoscope lens[36]

    下载: 全尺寸图片 幻灯片

    图 8  基于抛物面镜内窥镜设计图[38]

    Figure 8.  Design of endoscope based on convex parabolic mirrors[38]

    下载: 全尺寸图片 幻灯片

    图 9  基于复合透镜的仿生内窥镜原型[39]

    Figure 9.  Prototype of bionic endoscope based on compound lens[39]

    下载: 全尺寸图片 幻灯片

    图 10  紧凑双视场内窥镜的设计示意图[41]

    Figure 10.  Schematic diagram of compact dual-view endoscope design[41]

    下载: 全尺寸图片 幻灯片

    图 11  真实内窥镜图像视差图。（a）输入图像（左）；（b）SGBM视差图；（c）StereoNet视差图[59]

    Figure 11.  Disparity results of real endoscopic images. (a) Input image (left); (b) disparity map produced by the SGBM algorithm; (c) disparity map produced by StereoNet

    下载: 全尺寸图片 幻灯片

    图 12  整个胃器官重建三维点云图[60]

    Figure 12.  Reconstruction 3D point cloud of the whole stomach[60]

    下载: 全尺寸图片 幻灯片

    图 13  实验场景及全景三维点云图[66]

    Figure 13.  Experiment scene and full-view 3D point clouds

    下载: 全尺寸图片 幻灯片

    图 14  单目内窥镜SLAM框架[70]

    Figure 14.  Framework for monocular SLAM-endoscopy[70]

    下载: 全尺寸图片 幻灯片

    图 15  SfM重建三维点云图[71]

    Figure 15.  3D point cloud reconstructed by SfM[71]

    下载: 全尺寸图片 幻灯片

    图 16  （a）CapsoCam SV1示意图及 （b）SVI拍摄的图像

    Figure 16.  (a) Schematic diagram of CapsoCam SV1 capsule endoscope and (b) image captured by SV1

    下载: 全尺寸图片 幻灯片

    图 17  全景胶囊内镜下CD粘膜萎缩内镜特征[8]

    Figure 17.  Typical signs of mucosal atrophy detected by panoramic capsule endoscope[8]

    下载: 全尺寸图片 幻灯片

    图 18  传统结肠镜和FUSE拍摄的图像[73]

    Figure 18.  Images captured by traditional colonoscopy and FUSE[73]

    下载: 全尺寸图片 幻灯片

    图 19  （a）FUSE结肠镜; （ b）FUSE拍摄的图像

    Figure 19.  (a) FUSE colonoscope; (b) image captured by FUSE

    下载: 全尺寸图片 幻灯片

    表  1  不同全景镜头技术对比

    Table  1.   Comparison of different panoramic lens technologyies

    成像方式视场结构复杂度成本畸变特点
    自由曲面棱镜[33]中中中小多视角成像，
    视场不连续
    全景环形透镜[34]大高中小宽视场、高分辨率
    折反射成像
    系统[35-37]    		大低低大加工难度低
    仿生光学系统[39-40]小高高小分辨率高、
    成像均匀
    多系统组合[41]小高中小边缘视场成
    像质量高
    下载: 导出CSV

    表  2  常用商用胶囊内镜

    Table  2.   Common commercial capsule endoscopes

    产品厂家国家帧率摄像头
    数量    		视场角
    PillCam SB3Medtronic美国2~61156°
    PillCam Crohn’s
    Capsule    		Medtronic美国4~352336°
    EndoCapsule 10Olympus日本21160°
    MiroCam MC1600IntroMedic韩国61170°
    MiroCam MC2000IntroMedic韩国3（per
    camera）    		2340°
    OMOM RC100Jinshan中国2~81160°
    CapsoCam PlusCapso
    Vision    		美国12-204360°
    下载: 导出CSV

    表  3  商用结肠内镜

    Table  3.   Commercial colonoscopies

    产品厂家国家视场角摄像头
    个数    		实现方式
    CF-HQ290Olympus日本170°1超广角
    FUSEEndo Choice美国330°3多视角显示
    EWAVEOlympus日本232°3多摄像头拼接
    Third Eye
    Panoromic    		Avantis美国>300°3多视角显示
    下载: 导出CSV
    Baidu
    • 参考文献(76)
  • [1] 熊兴波, 樊淑梅, 江镇州, 等. 胶囊内镜在胃肠道疾病中的诊断价值研究[J]. 中国实用医药,2021,16(19):85-87. doi: 10.14163/j.cnki.11-5547/r.2021.19.030

    XIONG X B, FAN SH M, JIANG ZH ZH, et al. Research on the diagnostic value of capsule endoscopy in gastrointestinal diseases[J]. China Practical Medicine, 2021, 16(19): 85-87. (in Chinese) doi: 10.14163/j.cnki.11-5547/r.2021.19.030
    [2] 国擎. 腹腔镜辅助与开腹直肠癌根治术治疗直肠癌的近期效果对比[J]. 中国实用医药,2022,17(3):30-32. doi: 10.14163/j.cnki.11-5547/r.2022.03.010

    GUO Q. Comparison of the recent effect of laparoscopic-assisted resection and open radical resection of rectal cancer in the treatment of rectal cancer[J]. China Practical Medicine, 2022, 17(3): 30-32. (in Chinese) doi: 10.14163/j.cnki.11-5547/r.2022.03.010
    [3] 高志强. 内镜联合腹腔镜在胃肠道病变手术治疗中的临床应用[J]. 影像研究与医学应用,2022,6(1):188-190. doi: 10.3969/j.issn.2096-3807.2022.01.064

    GAO ZH Q. Clinical application of endoscopy combined with laparoscopy in the surgical treatment of gastrointestinal lesions[J]. Journal of Imaging Research and Medical Applications, 2022, 6(1): 188-190. (in Chinese) doi: 10.3969/j.issn.2096-3807.2022.01.064
    [4] VAN KEULEN K E, SOONS E, SIERSEMA P D. The role of behind folds visualizing techniques and technologies in improving adenoma detection rate[J]. Current Treatment Options in Gastroenterology, 2019, 17(3): 394-407. doi: 10.1007/s11938-019-00242-5
    [5] FRIEDRICH K, GEHRKE S, STREMMEL W, et al. First clinical trial of a newly developed capsule endoscope with panoramic side view for small bowel: a pilot study[J]. Journal of Gastroenterology and Hepatology, 2013, 28(9): 1496-1501. doi: 10.1111/jgh.12280
    [6] ZAMMIT S C, MCALINDON M E, SIDHU R. PTU-084 the use of panoramic capsule endoscopy in obscure gastrointestinal bleeding[J]. Gut, 2019, 68(S2): A233.
    [7] ZWINGER L L, SIEGMUND B, STROUX A, et al. CapsoCam SV-1 versus PillCam SB 3 in the detection of obscure gastrointestinal bleeding: results of a prospective randomized comparative multicenter study[J]. Journal of Clinical Gastroenterology, 2019, 53(3): e101-e106. doi: 10.1097/MCG.0000000000000994
    [8] BRANCHI F, FERRETTI F, ORLANDO S, et al. Small-bowel capsule endoscopy in patients with celiac disease, axial versus lateral/panoramic view: Results from a prospective randomized trial[J]. Digestive Endoscopy, 2020, 32(5): 778-784. doi: 10.1111/den.13575
    [9] NAYA Y, NAKAMURA K, ARAKI K, et al. Usefulness of panoramic views for novice surgeons doing retroperitoneal laparoscopic nephrectomy[J]. International Journal of Urology, 2009, 16(2): 177-180. doi: 10.1111/j.1442-2042.2008.02215.x
    [10] ELOSUA A, RULLAN M, RUBIO S, et al. Does capsule endoscopy impact clinical management in established Crohn's disease?[J]. Digestive and Liver Disease, 2022, 54(1): 118-124. doi: 10.1016/j.dld.2021.08.014
    [11] POPOVIC V, SEYID K, COGAL Ö, et al. . State-of-the-art multi-camera systems[M]//POPOVIC V, SEYID K, COGAL Ö, et al. . Design and Implementation of Real-Time Multi-Sensor Vision Systems. Cham: Springer, 2017: 13-31.
    [12] ZENG J L, CHENG Y H, WU T Y, et al. . MicroEYE: a wireless multiple-lenses panoramic endoscopic system[C]. International Conference on Advanced Engineering Theory and Applications, Springer, 2017: 190-200.
    [13] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110. doi: 10.1023/B:VISI.0000029664.99615.94
    [14] ROSTEN E, DRUMMOND T. Machine learning for high-speed corner detection[C]. 9th European Conference on Computer Vision, Springer, 2006: 430-443.
    [15] BAY H, TUYTELAARS T, VAN GOOL L. Surf: Speeded up robust features[C]. 9th European Conference on Computer Vision, Springer, 2006: 404-417.
    [16] RUBLEE E, RABAUD V, KONOLIGE K, et al. . ORB: An efficient alternative to SIFT or SURF[C]. International Conference on Computer Vision, Barcelona, 2011: 2564-2571.
    [17] MUJA M, LOWE D G. Fast approximate nearest neighbors with automatic algorithm configuration[C]. Proceedings of the Fourth International Conference on Computer Vision Theory and Applications, INSTICC Press, 2009.
    [18] FISCHLER M A, BOLLES R C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6): 381-395. doi: 10.1145/358669.358692
    [19] PENG C H, CHENG C H. A panoramic endoscope design and implementation for Minimally Invasive Surgery[C]. 2014 IEEE International Symposium on Circuits and Systems (ISCAS), IEEE, 2014: 453-456.
    [20] CHENG C H, PENG C H, GUO J I, et al. A two-lens minimally invasive surgical panoramic endoscope design and implementation[J]. International Journal of Electrical Engineering, 2014, 21(6): 235-241.
    [21] CHENG C H, HUNG SH P, GUO J I, et al. . A wireless panoramic endoscope system design and implementation for minimally invasive surgery[C]. 2015 IEEE International Symposium on Circuits and Systems (ISCAS), IEEE, 2015: 1895-1895.
    [22] KIM D T, CHENG C H. A panoramic stitching vision performance improvement technique for Minimally Invasive Surgery[C]. 2016 5th International Symposium on Next-Generation Electronics (ISNE), IEEE, 2016: 1-2.
    [23] KIM D T, NGUYEN V T, CHENG C H, et al. Speed improvement in image stitching for panoramic dynamic images during minimally invasive surgery[J]. Journal of Healthcare Engineering, 2018, 2018: 3654210.
    [24] KIM D T, CHENG C H, LIU D G, et al. Designing a new endoscope for panoramic-view with focus-area 3D-vision in minimally invasive sur