| Citation: | ZHENG Yue-kun, GE Ming-feng, CHANG Zhi-min, DONG Wen-fei. Super-resolution reconstruction for colorectal endoscopic images based on a residual network[J].Chinese Optics, 2023, 16(5): 1022-1033.doi:10.37188/CO.2022-0247
|
In this paper, an image super-resolution reconstruction multi-scale algorithm based on a residual attention network (SMRAN) is proposed to solve the problems caused by low resolutions, less texture information and blurred details in colorectal endoscopic images. Images from the colorectal polyp endoscope image dataset PolypsSet are selected as the raw data for these experiments. A convolutional network is built to extract the shallow features of the low-resolution image and a Res-Sobel block is designed to enhance its edge features. A multi-scale feature fusion block MEB is designed by introducing convolution kernels of different sizes to adaptively extract image features of different scales and obtain effective image information. The Res-Sobel block and multi-scale feature fusion module block MEB are connected through the residual attention network. Finally, a high-resolution image is reconstructed at the sub-pixel convolution layer. When the amplification factor is ×4, the performance of the proposed algorithm on the test set are as follows: the peak signal-to-noise ratio (PSNR) is 34.25 dB and the structural similarity (SSIM) is 0.8675. Compared with the traditional bicubic interpolation algorithm and commonly used deep learning algorithms such as SRCNN and RCAN, the proposed SMRAN algorithm shows better super-resolution reconstruction results on colorectal endoscopic images.
| [1] |
刘爽, 田兆星, 李浩然, 等. 一种基于改进YOLOv5s网络的结直肠腺瘤实时检测方法[J]. 河北大学学报(自然科学版),2022,42(3):327-336.
LIU SH, TIAN ZH X, LI H R,
et al. A real-time method for colorectal adenoma detection based on an improved YOLOv5s network[J].
Journal of Hebei University(
Natural Science Edition), 2022, 42(3): 327-336. (in Chinese)
|
| [2] |
白瑞峰, 江山, 孙海江, 等. 基于编码解码结构的微血管减压图像实时语义分割[J]. 中国光学(中英文),2022,15(5):1055-1065.
doi:10.37188/CO.2022-0120
BAI R F, JIANG SH, SUN H J,
et al. Real-time semantic segmentation of microvascular decompression images based on encoder-decoder structure[J].
Chinese Optics, 2022, 15(5): 1055-1065. (in Chinese)
doi:10.37188/CO.2022-0120
|
| [3] |
陈晓冬, 艾大航, 张佳琛, 等. Gabor滤波融合卷积神经网络的路面裂缝检测方法[J]. 中国光学,2020,13(6):1293-1301.
doi:10.37188/CO.2020-0041
CHEN X D, AI D H, ZHANG J CH,
et al. Gabor filter fusion network for pavement crack detection[J].
Chinese Optics, 2020, 13(6): 1293-1301. (in Chinese)
doi:10.37188/CO.2020-0041
|
| [4] |
左超, 陈钱. 分辨率、超分辨率与空间带宽积拓展——从计算光学成像角度的一些思考[J]. 中国光学(中英文),2022,15(6):1105-1166.
doi:10.37188/CO.2022-0105
ZUO CH, CHEN Q. Resolution, super-resolution and spatial bandwidth product expansion——some thoughts from the perspective of computational optical imaging[J].
Chinese Optics, 2022, 15(6): 1105-1166. (in Chinese)
doi:10.37188/CO.2022-0105
|
| [5] |
GU X G, ZHOU F X, CHEN R F,
et al. . Endoscopic single image super-resolution based on transformer and convolutional neural network[C].
7th International Conference on Intelligent Life System Modelling, Image Processing and Analysis, Springer, 2021: 24-32.
|
| [6] |
YANG X R, CHEN Y, TAO R,
et al. . Endoscopic image deblurring and super-resolution reconstruction based on deep learning[C].
2020 International Conference on Artificial Intelligence and Computer Engineering (ICAICE), IEEE, 2020: 168-172.
|
| [7] |
TURAN M. A generative adversarial network based super-resolution approach for capsule endoscopy images[J].
Medicine Science, 2021, 10(3): 1002-1007.
doi:10.5455/medscience.2021.06.218
|
| [8] |
TAŞ M, YILMAZ B. Super resolution convolutional neural network based pre-processing for automatic polyp detection in colonoscopy images[J].
Computers&
Electrical Engineering, 2021, 90: 106959.
|
| [9] |
DONG CH, LOY C C, HE K M,
et al. Image super-resolution using deep convolutional networks[J].
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307.
doi:10.1109/TPAMI.2015.2439281
|
| [10] |
雷茂, 郭锋, 秦明伟. 基于改进POCS算法的太赫兹图像超分辨率重建[J]. 传感器与微系统,2022,41(3):122-125.
doi:10.13873/J.1000-9787(2022)03-0122-04
LEI M, GUO F, QIN M W. Super-resolution reconstruction for terahertz images based on improved POCS algorithm[J].
Transducer and Microsystem Technologies, 2022, 41(3): 122-125. (in Chinese)
doi:10.13873/J.1000-9787(2022)03-0122-04
|
| [11] |
孙琰玥, 何小海, 陈为龙. 小波局部适应插值的图像超分辨率重建[J]. 计算机工程,2010,36(13):183-185.
doi:10.3969/j.issn.1000-3428.2010.13.065
SUN Y Y, HE X H, CHEN W L. Image super-resolution reconstruction of wavelet local-adaptation interpolation[J].
Computer Engineering, 2010, 36(13): 183-185. (in Chinese)
doi:10.3969/j.issn.1000-3428.2010.13.065
|
| [12] |
庞博, 张旭东, 徐小红. 自适应图像插值在超分辨率图像重建中的应用[J]. 合肥工业大学学报(自然科学版),2006,29(7):825-829.
PANG B, ZHANG X D, XU X H. Super-resolution image reconstruction using adaptive interpolation method[J].
Journal of Hefei University of Technology(
Natural Science), 2006, 29(7): 825-829. (in Chinese)
|
| [13] |
陈楠, 张标. 多尺度半耦合卷积稀疏编码的遥感影像超分辨率重建[J]. 计算机辅助设计与图形学学报,2022,34(3):382-391.
CHEN N, ZHANG B. Multi-scale semi-coupled convolutional sparse coding for the super-resolution reconstruction of remote sensing image[J].
Journal of Computer-Aided Design&
Computer Graphics, 2022, 34(3): 382-391. (in Chinese)
|
| [14] |
李方玗, 贾晓芬, 赵佰亭, 等. 高效多注意力特征融合的图像超分辨率重建算法[J/OL]. 小型微型计算机系统: 1-9[2022-11-25]. http://kns.cnki.net/kcms/detail/21.1106.TP.20221111.1852.006.html.
LI F Y, JIA X F, ZHAO B T,
et al. . Efficient multi-attention feature fusion for image super-resolution reconstruction algorithms[J/OL].
Journal of Chinese Computer Systems: 1-9[2022-11-25]. http://kns.cnki.net/kcms/detail/21.1106.TP.20221111.1852.006.html. (in Chinese)
|
| [15] |
林敏强, 赵曈, 祝明欣. 基于残差注意力的合成孔径雷达图像超分辨率算法[J]. 电子产品世界,2022,29(11):33-34.
LIN M Q, ZHAO T, ZHU M X. SAR image super-resolution algorithm based on residual attention[J].
Electronic Engineering&
Product World, 2022, 29(11): 33-34. (in Chinese)
|
| [16] |
须颖, 刘帅, 邵萌, 等. 一种多尺度GAN的低剂量CT超分辨率重建方法[J]. 西安电子科技大学学报,2022,49(2):228-236.
doi:10.19665/j.issn1001-2400.2022.02.026
XU Y, LIU SH, SHAO M,
et al. Multi-scale generation antagonistic network for the low-dose CT images super-resolution reconstruction algorithm[J].
Journal of Xidian University, 2022, 49(2): 228-236. (in Chinese)
doi:10.19665/j.issn1001-2400.2022.02.026
|
| [17] |
LIM B, SON S, KIM H,
et al. . Enhanced deep residual networks for single image super-resolution[C].
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, IEEE, 2017: 1132-1140.
|
| [18] |
KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C].
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2016: 1646-1654.
|
| [19] |
ZHANG Y L, LI K P, LI K,
et al. . Image super-resolution using very deep residual channel attention networks[C].
Proceedings of the 15th European Conference on Computer Vision, Springer, 2018: 294-310.
|
| [20] |
LEDIG C, THEIS L, HUSZÁR F,
et al. . Photo-realistic single image super-resolution using a generative adversarial network[C].
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2017: 105-114.
|
| [21] |
SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C].
3rd International Conference on Learning Representations, ICLR, 2015.
|
| [22] |
王玉玺, 陈健美. 基于自适应Canny算子和多方向Sobel算子的虹膜边缘检测算法[J]. 计算机与数字工程,2020,48(11):2744-2749.
doi:10.3969/j.issn.1672-9722.2020.11.040
WANG Y X, CHEN J M. Iris edge detection algorithm based on adaptive canny operator and multi-directional Sobel operator[J].
Computer&
Digital Engineering, 2020, 48(11): 2744-2749. (in Chinese)
doi:10.3969/j.issn.1672-9722.2020.11.040
|
| [23] |
LIU S R, TANG X Y, WANG D. Facial expression recognition based on Sobel operator and improved CNN-SVM[C].
2020 IEEE 3rd International Conference on Information Communication and Signal Processing (ICICSP), IEEE, 2020: 236-240.
|
| [24] |
HAO F, XU D SH, CHEN D L,
et al. Sobel operator enhancement based on eight-directional convolution and entropy[J].
International Journal of Information Technology, 2021, 13(5): 1823-1828.
doi:10.1007/s41870-021-00770-3
|
| [25] |
WOO S, PARK J, LEE J Y,
et al. . CBAM: convolutional block attention module[C].
Proceedings of the 15th European Conference on Computer Vision (ECCV), Springer, 2018: 3-19.
|
| [26] |
SHI W ZH, CABALLERO J, HUSZÁR F,
et al. . Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C].
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2016: 1874-1883.
|
| [27] |
LI K D, FATHAN M I, PATEL K,
et al. Colonoscopy polyp detection and classification: Dataset creation and comparative evaluations[J].
PLoS One, 2021, 16(8): e0255809.
doi:10.1371/journal.pone.0255809
|
| [28] |
JHA D, SMEDSRUD P H, RIEGLER M A,
et al. . Kvasir-SEG: a segmented polyp dataset[C].
26th International Conference on Multimedia Modeling, Springer, 2020: 451-462.
|
Figures(9)/Tables(6)
DownLoad:
