| Citation: | CHEN Chun-yi, WU Xin-yi, HU Xiao-juan, YU Hai-yang. Image super-resolution reconstruction with multi-scale attention fusion[J].Chinese Optics, 2023, 16(5): 1034-1044.doi:10.37188/CO.2023-0020
|
The resolution of optical imaging is limited by the diffraction limit, system detector size and many other factors. To obtain images with richer details and clearer textures, a multi-scale feature attention fusion residual network was proposed. Firstly, shallow features of the image were extracted using a layer of convolution and then the multi-scale features were extracted by a cascade of multi-scale feature extraction units. The local channel attention module is introduced in the multi-scale feature extraction unit to adaptively correct the weights of feature channels and improve the attention to high frequency information. The shallow features and the output of each multi-scale feature extraction unit were used as hierarchical features for global feature fusion reconstruction. Finally, the hight-resolution image was reconstructed by introducing shallow features and multi-level image features using the residual branch. Charbonnier loss was adopted to make the training more stable and converge faster. Comparative experiments on the international benchmark datasets show that the model outperforms most state-of-the-art methods on objective metrics. Especially on the Set5 data set, the PSNR index of the 4× reconstruction result is increased by 0.39 dB, and the SSIM index is increased to 0.8992, and the subjective visual effect of the algorithm is better.
| [1] |
左超, 陈钱. 分辨率、超分辨率与空间带宽积拓展—从计算光学成像角度的一些思考[J]. 中国光学(中英文),2022,15(6):1105-1166.
ZUO CH, CHEN Q. Resolution, super-resolution and spatial bandwidth product expansion——some thoughts from the perspective of computational optical imaging[J].
Chinese Optics, 2022, 15(6): 1105-1166. (in Chinese)
|
| [2] |
吴靖, 叶晓晶, 黄峰, 等. 基于深度学习的单帧图像超分辨率重建综述[J]. 电子学报,2022,50(9):2265-2294.
WU J, YE X J, HUANG F,
et al. A review of single image super-resolution reconstruction based on deep learning[J].
Acta Electronica Sinica, 2022, 50(9): 2265-2294. (in Chinese)
|
| [3] |
李洪安, 郑峭雪, 陶若霖, 等. 基于深度学习的图像超分辨率研究综述[J]. 图学学报,2023,44(1):1-15.
LI H A, ZHENG Q X, TAO R L,
et al. Review of image super-resolution based on deep learning[J].
Journal of Graphics, 2023, 44(1): 1-15. (in Chinese)
|
| [4] |
毕勇, 潘鸣奇, 张硕, 等. 三维点云数据超分辨率技术[J]. 中国光学(中英文),2022,15(2):210-223.
BI Y, PAN M Q, ZHANG SH,
et al. Overview of 3D point cloud super-resolution technology[J].
Chinese Optics, 2022, 15(2): 210-223. (in Chinese)
|
| [5] |
王溢琴, 董云云, 刘慧玲. 基于GoogLeNet和空间谱变换的高光谱图像超分辨率方法[J]. 光学技术,2022,48(1):93-101.
doi:10.3321/j.issn.1002-1582.2022.1.gxjs202201015
WANG Y Q, DONG Y Y, LIU H L. Super-resolution method of hyperspectral image based on GoogLeNet and spatial spectrum transformation[J].
Optical Technique, 2022, 48(1): 93-101. (in Chinese)
doi:10.3321/j.issn.1002-1582.2022.1.gxjs202201015
|
| [6] |
曲海成, 王雅萱, 申磊. 多感受野特征与空谱注意力结合的高光谱图像超分辨率算法[J]. 自然资源遥感,2022,34(1):43-52.
QU H CH, WANG Y X, SHEN L. Hyperspectral super-resolution combining multi-receptive field features with spectral-spatial attention[J].
Remote Sensing for Natural Resources, 2022, 34(1): 43-52. (in Chinese)
|
| [7] |
柯舒婷, 陈明惠, 郑泽希, 等. 生成对抗网络对OCT视网膜图像的超分辨率重建[J]. 中国 ,2022,49(15):1507203.
KE SH T, CHEN M H, ZHENG Z X,
et al. Super-resolution reconstruction of optical coherence tomography retinal images by generating adversarial network[J].
Chinese Journal of Lasers, 2022, 49(15): 1507203. (in Chinese)
|
| [8] |
左艳, 黄钢, 聂生东. 深度学习在医学影像智能处理中的应用与挑战[J]. 中国图象图形学报,2021,26(2):305-315.
doi:10.11834/jig.190470
ZUO Y, HUANG G, NIE SH D. Application and challenges of deep learning in the intelligent processing of medical images[J].
Journal of Image and Graphics, 2021, 26(2): 305-315. (in Chinese)
doi:10.11834/jig.190470
|
| [9] |
王一宁, 赵青杉, 秦品乐, 等. 基于轻量密集神经网络的医学图像超分辨率重建算法[J]. 计算机应用,2022,42(8):2586-2592.
WANG Y N, ZHAO Q SH, QIN P L,
et al. Super-resolution reconstruction algorithm of medical image based on lightweight dense neural network[J].
Journal of Computer Applications, 2022, 42(8): 2586-2592. (in Chinese)
|
| [10] |
耿铭昆, 吴凡路, 王栋. 轻量化火星遥感影像超分辨率重建网络[J]. 光学 精密工程,2022,30(12):1487-1498.
doi:10.37188/OPE.20223012.1487
GENG M K, WU F L, WANG, D. Lightweight Mars remote sensing image super-resolution reconstruction network[J].
Optics and Precision Engineering, 2022, 30(12): 1487-1498. (in Chinese)
doi:10.37188/OPE.20223012.1487
|
| [11] |
ZHANG J ZH, XU T F, LI J N,
et al. Single-image super resolution of remote sensing images with real-world degradation modeling[J].
Remote Sensing, 2022, 14(12): 2895.
doi:10.3390/rs14122895
|
| [12] |
倪若婷, 周莲英. 基于卷积神经网络的人脸图像超分辨率重建方法[J]. 计算机与数字工程,2022,50(1):195-200.
doi:10.3969/j.issn.1672-9722.2022.01.037
NI R T, ZHOU L Y. Face image super-resolution reconstruction method based on convolutional neural network[J].
Computer&
Digital Engineering, 2022, 50(1): 195-200. (in Chinese)
doi:10.3969/j.issn.1672-9722.2022.01.037
|
| [13] |
卢峰, 周琳, 蔡小辉. 面向安防监控场景的低分辨率人脸识别算法研究[J]. 计算机应用研究,2021,38(4):1230-1234.
doi:10.19734/j.issn.1001-3695.2020.01.0074
LU F, ZHOU L, CAI X H. Research on low-resolution face recognition algorithm for security surveillance scene[J].
Application Research of Computers, 2021, 38(4): 1230-1234. (in Chinese)
doi:10.19734/j.issn.1001-3695.2020.01.0074
|
| [14] |
KEYS R. Cubic convolution interpolation for digital image processing[J].
IEEE Transactions on Acoustics,
Speech,
and Signal Processing, 1981, 29(6): 1153-1160.
doi:10.1109/TASSP.1981.1163711
|
| [15] |
黄友文, 唐欣, 周斌. 结合双注意力和结构相似度量的图像超分辨率重建网络[J]. 液晶与显示,2022,37(3):367-375.
doi:10.37188/CJLCD.2021-0178
HUANG Y W, TANG X, ZHOU B. Image super-resolution reconstruction network with dual attention and structural similarity measure[J].
Chinese Journal of Liquid Crystals and Displays, 2022, 37(3): 367-375. (in Chinese)
doi:10.37188/CJLCD.2021-0178
|
| [16] |
周乐, 徐龙, 刘孝艳, 等. 基于梯度感知的单幅图像超分辨[J]. 液晶与显示,2022,37(10):1334-1344.
doi:10.37188/CJLCD.2022-0083
ZHOU L, XU L, LIU X Y,
et al. Gradient-aware based single image super-resolution[J].
Chinese Journal of Liquid Crystals and Displays, 2022, 37(10): 1334-1344. (in Chinese)
doi:10.37188/CJLCD.2022-0083
|
| [17] |
DONG CH, LOY C C, HE K M,
et al.. Learning a deep convolutional network for image super-resolution[C].
Proceedings of the 13th European Conference on Computer Vision, Springer, 2014: 184-199.
|
| [18] |
DONG CH, LOY C C, TANG X O. Accelerating the super-resolution convolutional neural network[C].
Proceedings of the 14th European Conference on Computer Vision, Springer, 2016: 391-407.
|
| [19] |
KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C].
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2016: 1646-1654.
|
| [20] |
TAI Y, YANG J, LIU X M. Image super-resolution via deep recursive residual network[C].
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2017: 2790-2798.
|
| [21] |
SHI W ZH, CABALLERO J, HUSZÁR F,
et al.. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C].
2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2016: 1874-1883.
|
| [22] |
ZHANG Y L, TIAN Y P, KONG Y,
et al. . Residual dense network for image super-resolution[C].
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2018: 2472-2481.
|
| [23] |
LIM B, SON S, KIM H,
et al. . Enhanced deep residual networks for single image super-resolution[C].
2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE, 2017: 1132-1140.
|
| [24] |
ZHANG Y L, LI K P, LI K,
et al. . Image super-resolution using very deep residual channel attention networks[C].
Proceedings of the 15th European Conference on Computer Vision, Springer, 2018: 294-310.
|
| [25] |
LI J CH, FANG F M, MEI K F,
et al. . Multi-scale residual network for image super-resolution[C].
Proceedings of the 15th European Conference on Computer Vision, Springer, 2018: 527-542.
|
| [26] |
ZHAO H Y, KONG X T, HE J W,
et al. . Efficient image super-resolution using pixel attention[C].
Proceedings of the European Conference on Computer Vision, Springer, 2020: 56-72.
|
| [27] |
WANG Y. Edge-enhanced feature distillation network for efficient super-resolution[C].
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE, 2022: 776-784.
|
| [28] |
LAI W SH, HUANG J B, AHUJA N,
et al.. Deep Laplacian pyramid networks for Fast and accurate super-resolution[C].
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2017: 5835-5843.
|
| [29] |
SZEGEDY C, LIU W, JIA Y Q,
et al.. Going deeper with convolutions[C].
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2015: 1-9.
|
| [30] |
HUI ZH, WANG X M, GAO X B. Fast and accurate single image super-resolution via information distillation network[C].
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2018: 723-731.
|
Figures(8)/Tables(5)
DownLoad:
