面部视频非接触式生理参数感知 -

姓名
邮箱
手机号码
标题
留言内容
验证码

面部视频非接触式生理参数感知

doi:10.37188/CO.2021-0157

长春理工大学生命科学技术学院，吉林长春 130022

基金项目:吉林省科技发展计划项目（No. 20210204131YY）

详细信息

作者简介:
嵇晓强（1982—），女，吉林德惠人，博士，副教授，硕士生导师，2012 年于中国科学院长春光学精密机械与物理研究所获得光学工程博士学位，主要从事医学信号及图像处理方面的研究。E-mail:zuoanmulan@163.com
刘振瑶（1997—），女，山东济宁人，硕士研究生，2019年于长春理工大学获得学士学位，主要从事医学信号及图像处理方面的研究。E-mail:1011510854@qq.com

中图分类号:TN911.7;TP391
计量
- 文章访问数:1094
- HTML全文浏览量:638
- PDF下载量:138
- 被引次数:0
出版历程
- 收稿日期:2021-08-12
- 修回日期:2021-09-14
- 网络出版日期:2021-10-19
- 刊出日期:2022-03-21

Non-contact perception of physiological parameters from videos of faces

School of Life Science and Technology, Changchun University of Science and Technology, Changchun 130022, China

Funds:Supported by Department of Science and Technology of Natural Science Foundation of Jilin Province under grant number (No. 20210204131YY)

More Information

Corresponding author:zuoanmulan@163.com

摘要

摘要:为了在非接触条件下检测受试者的各项生理参数，本文设计了一种基于成像式光电容积描记技术，从手机录制的人脸视频中估算生理参数的方法。首先，提出了“小波变换-主成分分析-盲源分离”算法，用于提取出高信噪比的RGB三通道脉搏波信号。然后，分别从频域和时域角度对绿色通道信号进行处理，估算出心率值和呼吸率值；对红蓝通道的脉搏波信号进行处理，并结合血氧仪检测的血氧饱和度结果，进行数据拟合，从而找到从面部视频中估算血氧饱和度值的最佳线性方程。最后，对比了自然光下各生理参数的估算结果误差，分析了在3种光照环境下各参数的估算结果。结果表明：3种光照环境下得到的心率平均误差为0.5512次/min，呼吸率平均误差为−0.6321次/min，血氧饱和度平均误差为−0.2743%。综上，本文提出的非接触式生理参数估算方法精度高，具有普适性和稳定性，估算结果同标准仪器的测量结果具有高度一致性，可满足日常生理参数测量的需求。
- 成像式光电容积描记技术/
- 非接触式/
- 小波变换-主成分分析-盲源分离/
- 心率/
- 呼吸率/
- 血氧饱和度
Abstract:Non-contact detection of various physiological parameters has attract great attention. In this paper, a method of estimating physiological parameters based on imaging photoplethysmography from videos of people’s faces recorded by mobile phone is proposed. First, a "wavelet transform-principal component analysis-blind source separation" algorithm is proposed to extract the video’s RGB three-channel pulse wave signal with a high signal-to-noise ratio. Then, the green channel signal is processed separately in the frequency and the time domains to estimate heart and respiratory rates. The pulse wave signals of the red and blue channels are processed, and combined with the oxygen saturation detected by an oximeter to perform data fitting, the best linear equation for estimating the oxygen saturation value from the facial video is found. Finally, the error of the estimation results of various physiological parameters under natural light is compared, and the estimation results of each parameter under three lighting environments are analyzed. The results show that under the three lighting environments, the average error of heart rate detection is 0.5512 time/min, the average error of respiration rate is −0.6321 time/min , and the average error of oxygen saturation is −0.2743%. In summary, the non-contact physiological parameter estimation method proposed in this paper is highly accurate, universally applicable and stable. Its estimation results are highly consistent with the measurement result of standard instruments, which meets the needs of daily physiological parameter measurement.
- imaging photoplethysmography/
- non-contact/
- wavelet transform-principal component analysis-blind signal separation/
- heart rate/
- respiratory rate/
- oxygen saturation

HTML全文

图 1视频图像处理流程图

Figure 1.Flow chart of video image processing

下载: 全尺寸图片幻灯片

图 2放大前、后IPPG源信号对比

Figure 2.Comparison of IPPG source signals before and after amplification

下载: 全尺寸图片幻灯片

图 3IPPG源信号处理流程图

Figure 3.Flow chart of IPPG source signal processing

下载: 全尺寸图片幻灯片

图 4PCA降维后各成分信号

Figure 4.The signal of each component after PCA dimensionality reduction

下载: 全尺寸图片幻灯片

图 5盲源分离出的独立源信号

Figure 5.Independent source signal separated by a blind source

下载: 全尺寸图片幻灯片

图 6经带通滤波后的脉搏波信号

Figure 6.Pulse wave signal after bandpass filtering

下载: 全尺寸图片幻灯片

图 7傅立叶变换频谱图

Figure 7.Fourier transform spectrogram

下载: 全尺寸图片幻灯片

图 8滤波后呼吸信号

Figure 8.Respiration signal after filtering

下载: 全尺寸图片幻灯片

图 9实验采集示意图及装置图

Figure 9.Schematic diagram and real diagram of the experimental acquisition device

下载: 全尺寸图片幻灯片

图 103种场景下心率估算结果比较

Figure 10.Comparison of heart rate estimation results in three scenarios

下载: 全尺寸图片幻灯片

图 113种场景下心率结果Bland-Altman 一致性分析

Figure 11.Bland-Altman consistency analysis of heart rate results in three scenarios

下载: 全尺寸图片幻灯片

图 123种场景下呼吸率估算结果比较

Figure 12.Comparison of respiratory rate estimation results in three scenarios

下载: 全尺寸图片幻灯片

图 133种场景下呼吸率结果Bland-Altman 一致性分析

Figure 13.Bland-Altman consistency analysis of respiratory rate results in three scenarios

下载: 全尺寸图片幻灯片

图 143种场景下SpO2估算结果比较

Figure 14.Comparison of SpO2 estimation results in three scenarios

下载: 全尺寸图片幻灯片

图 153种场景下SpO2结果Bland–Altman 一致性分析

Figure 15.Bland-Altman consistency analysis of SpO2 results in three scenarios

下载: 全尺寸图片幻灯片

表 1自然光下各心率检测算法性能比较

Table 1.Performance comparison of various heart rate detection algorithms under natural light

方法	\|Me\| (time·min⁻¹)	SDe (time·min⁻¹)	RMSE (time·min⁻¹)	Mer	Cor
文献[6]	1.45	1.94	2.06	1.95%	0.9278
文献[12]	2.39	3.56	3.38	3.05%	—
本文方法	1.78	1.77	1.98	2.54%	0.9668

下载: 导出CSV

表 2自然光下各呼吸率检测算法性能比较

Table 2.Performance comparison of various respiratory rate detection algorithms under natural light

方法	Me (time·min⁻¹)	\|Me\| (time·min⁻¹)	SDe (time·min⁻¹)	RMSE (time/min⁻¹)	Cor
文献[10]	−0.58	2.54	3.98	4.02	0.61
本文方法	0.63	1.78	1.88	1.98	0.60

下载: 导出CSV

表 3自然光下各SpO2检测算法性能比较

Table 3.Performance comparison of SpO2 detection algorithms under natural light

方法	Me	SDe	RMSE	Mer
文献[8]	0.043%	1.10%	2.02%	2.7%
文献[11]	1.00%	1.32%	0.87%	—
本文方法	−0.27%	1.05%	1.08%	0.82%

下载: 导出CSV

参考文献 (27)

[1]	TULPPO M P, KIVINIEMI A M, JUNTTILA M J,et al. Home monitoring of heart rate as a predictor of imminent cardiovascular events[J].Frontiers in Physiology, 2019, 10: 341.doi:10.3389/fphys.2019.00341
[2]	KAUSER M A, SAFIUDDIN M. Heart rate-an emerging cardiovascular risk factor[J].University Heart Journal, 2012, 8(1): 30-35.doi:10.3329/uhj.v8i1.11665
[3]	MATHER M, THAYER J F. How heart rate variability affects emotion regulation brain networks[J].Current Opinion in Behavioral Sciences, 2018, 19: 98-104.doi:10.1016/j.cobeha.2017.12.017
[4]	孙莉娜, 李䶮, 郭汉涛, 等. 氮、铁共掺杂碳纳米粒子的制备及在过氧化氢和葡萄糖检测中的应用[J]. 应用化学,2020,37(3):350-358. SUN L N, LI Y, GUO H T,et al. Preparation of nitrogen and iron co-doped carbon nanoparticles and their applications in detection of hydrogen peroxide and glucose[J].Chinese Journal of Applied Chemistry, 2020, 37(3): 350-358. (in Chinese)
[5]	TARASSENKO L, VILLARROEL M, GUAZZI A,et al. Non-contact video-based vital sign monitoring using ambient light and auto-regressive models[J].Physiological Measurement, 2014, 35(5): 807-831.doi:10.1088/0967-3334/35/5/807
[6]	BAL U. Non-contact estimation of heart rate and oxygen saturation using ambient light[J].Biomedical Optics Express, 2015, 6(1): 86-97.doi:10.1364/BOE.6.000086
[7]	CASALINO G, CASTELLANO G, PASQUADIBISCEGLIE V,et al. Contact-less real-time monitoring of cardiovascular risk using video imaging and fuzzy inference rules[J].Information, 2019, 10(1): 9.
[8]	荣猛, 范强, 李凯扬. 基于IPPG非接触式生理参数测量算法的研究[J]. 生物医学工程研究,2018,37(1):27-31,35. RONG M, FAN Q, LI K Y. Study on the measurement algorithm of contactless physiological parameter based on imaging photoplenthysmography[J].Journal of Biomedical Engineering Research, 2018, 37(1): 27-31,35. (in Chinese)
[9]	李晓媛, 武鹏, 刘允, 等. 基于人脸视频的心率参数提取[J]. 光学精密工程,2020,28(3):548-557.doi:10.3788/OPE.20202803.0548 LI X Y, WU P, LIU Y,et al. Extraction of heart rate parameters from video of human face[J].Optics and Precision Engineering, 2020, 28(3): 548-557. (in Chinese)doi:10.3788/OPE.20202803.0548
[10]	LUGUERN D, MACWAN R, BENEZETH Y,et al. Wavelet variance maximization: a contactless respiration rate estimation method based on remote photoplethysmography[J].Biomedical Signal Processing and Control, 2021, 63: 102263.doi:10.1016/j.bspc.2020.102263
[11]	RAHMAN H, AHMED M U, BEGUM S. Non-contact physiological parameters extraction using facial video considering illumination, motion, movement and vibration[J].IEEE Transactions on Biomedical Engineering, 2020, 67(1): 88-98.doi:10.1109/TBME.2019.2908349
[12]	戴阳, 郑婷婷, 杨雪. 基于视频放大与盲源分离的非接触式心率检测[J]. 计算机系统应用,2021,30(1):228-234. DAI Y, ZHENG T T, YANG X. Non-contact heart rate detection based on video amplification and blind source separation[J].Computer Systems&Applications, 2021, 30(1): 228-234. (in Chinese)
[13]	CHEN Q, JIANG X Y, LIU X Y,etal.. Non-contact heart rate monitoring in neonatal intensive care unit using RGB camera[C].Proceedingsofthe42ndAnnualInternationalConferenceoftheIEEEEngineeringinMedicine&BiologySociety,IEEE, 2020: 5822-5825.
[14]	WU H Y, RUBINSTEIN M, SHIH E,et al. Eulerian video magnification for revealing subtle changes in the world[J].ACM Transactions on Graphics, 2012, 31(4): 65.
[15]	田昕, 赖翰文, 刘亚栋, 等. 基于卷积运算的嵌段共聚物自组装图像缺陷分析统计方法[J]. 应用化学,2021,38(9):1199-1208. TIAN X, LAI H W, LIU Y D,et al. Analysis of defects in block copolymer films by a convolution algorithm[J].Chinese Journal of Applied Chemistry, 2021, 38(9): 1199-1208. (in Chinese)
[16]	LIU J G, LUO H, ZHENG P P. et al. Transdermal optical imaging revealed different spatiotemporal patterns of facial cardiovascular activities[J].Scientific Reports, 2018, 8(1): 10588.doi:10.1038/s41598-018-28804-0
[17]	彭仁杰, 姚云霞. 小波分解和重构的光学可变图像融合研究[J]. 杂志,2021,42(3):145-148. PENG R J, YAO Y X. Research on optical variable image fusion based on wavelet decomposition and reconstruction[J].Laser Journal, 2021, 42(3): 145-148. (in Chinese)
[18]	YING L, ZHANG F B, ZHANG Y L,et al. Identifying distributed dynamic loading in one spatial dimension based on combing wavelet decomposition and Kalman filter with unknown input[J].Journal of Aerospace Engineering, 2021, 34(4): 04021025.doi:10.1061/(ASCE)AS.1943-5525.0001265
[19]	SONG K, ZHANG B Y, LI W,et al. Research on parallel principal component analysis based on ternary optical computer[J].Optik, 2021, 241: 167176.doi:10.1016/j.ijleo.2021.167176
[20]	张申华, 杨延西, 秦峤孟. 针对光栅图像的快速盲去噪方法[J]. 中国光学,2021,14(3):596-604.doi:10.37188/CO.2020-0166 ZHANG SH H, YANG Y X, QIN Q M. A fast blind denoising method for grating image[J].Chinese Optics, 2021, 14(3): 596-604. (in Chinese)doi:10.37188/CO.2020-0166
[21]	邱晓华, 李敏, 邓光芒, 等. 多层卷积特征融合的双波段决策级船舶识别[J]. 光学精密工程,2021,29(1):183-190.doi:10.37188/OPE.20212901.0183 QIU X H, LI M, DENG G M,et al. Multi-layer convolutional features fusion for dual-band decision-level ship recognition[J].Optics and Precision Engineering, 2021, 29(1): 183-190. (in Chinese)doi:10.37188/OPE.20212901.0183
[22]	VERKRUYSSE W, SVAASAND L O, NELSON J S. Remote plethysmographic imaging using ambient light[J].Optics Express, 2008, 16(26): 21434-21445.doi:10.1364/OE.16.021434
[23]	乔闹生, 孙萍. CCD非线性效应对双频光栅三维面形测量的影响[J]. 中国光学,2021,14(3):661-669.doi:10.37188/CO.2020-0143 QIAO N SH, SUN P. Influence of CCD nonlinearity effect on the three-dimensional shape measurement of dual frequency grating[J].Chinese Optics, 2021, 14(3): 661-669. (in Chinese)doi:10.37188/CO.2020-0143
[24]	赵云, 吕金光, 秦余欣, 等. 微型傅立叶变换光谱仪的优化设计与实验研究[J]. 中国光学,2020,13(2):411-425.doi:10.3788/co.20201302.0411 ZHAO Y, LV J G, QIN Y X,et al. Optimization design and experimental study of micro-Fourier transform spectrometer[J].Chinese Optics, 2020, 13(2): 411-425. (in Chinese)doi:10.3788/co.20201302.0411
[25]	孙学辉, 赵冰, 骆震, 等. 离散傅立叶变换用于非连续工业数据分析[J]. 分析化学,2020,48(10):1422-1427. SUN X H, ZHAO B, TUO Z,et al. Non-continuous industrial data Analysis using discrete fourier fransform[J].Chinese Journal of Analytical Chemistry, 2020, 48(10): 1422-1427. (in Chinese)
[26]	刘今越, 刘浩, 贾晓辉, 等. 基于视觉的非接触呼吸频率自动检测方法[J]. 仪器仪表学报,2019,40(2):51-58. LIU J Y, LIU H, JIA X H,et al. Vision-based automatic detection method for non-contact respiratory rate[J].Chinese Journal of Scientific Instrument, 2019, 40(2): 51-58. (in Chinese)
[27]	任繁栋, 丁筱雪, 蔡芳, 等. 基于超高效液相色谱-高分辨质谱联用技术研究冠心病及冠心病合并2型糖尿病患者代谢特征[J]. 分析化学,2020,48(1):49-56. REN F D, DING X X, CAI F,et al. Investigation of metabolic features of patients with coronary heart disease or coronary heart disease-type 2 diabetes mellitus based on ultra-high performance liquid chromatography high resolution mass spectrometry[J].Chinese Journal of Analytical Chemistry, 2020, 48(1): 49-56. (in Chinese)