Heartbeat and Respiration Rate Prediction Using Combined Photoplethysmography and Ballisto Cardiography

Owing to the recent trends in remote health monitoring,real-time appli-cations for measuring Heartbeat Rate and Respiration Rate(HARR)from video signals are growing rapidly.Photo Plethysmo Graphy(PPG)is a method that is operated by estimating the infinitesimal change in color of the human face,rigid motion of facial skin and head parts,etc.Ballisto Cardiography(BCG)is a non-surgical tool for obtaining a graphical depiction of the human body’s heartbeat by inducing repetitive movements found in the heart pulses.The resilience against motion artifacts induced by luminancefluctuation and the patient’s mobility var-iation is the major difficulty faced while processing the real-time video signals.In this research,a video-based HARR measuring framework is proposed based on combined PPG and BCG.Here,the noise from the input video signals is removed by using an Adaptive Kalmanfilter(AKF).Three different algorithms are used for estimating the HARR from the noise-free input signals.Initially,the noise-free sig-nals are subjected to Modified Adaptive Fourier Decomposition(MAFD)and then to Enhanced Hilbert vibration Decomposition(EHVD)andfinally to Improved Var-iation mode Decomposition(IVMD)for attaining three various results of HARR.The obtained values are compared with each other and found that the EHVD is showing better results when compared with all the other methods.

Blood Pressure and Heart Rate Measurements Using Photoplethysmography with Modified LRCN

In this study,single-channel photoplethysmography(PPG)signals were used to estimate the heart rate(HR),diastolic blood pressure(DBP),and systolic blood pressure(SBP).A deep learning model was proposed using a long-term recurrent convolutional network(LRCN)modified from a deep learning algorithm,the convolutional neural network model of the modified inception deep learning module,and a long short-term memory network(LSTM)to improve the model’s accuracy of BP and HR measurements.The PPG data of 1,551 patients were obtained from the University of California Irvine Machine Learning Repository.How to design a filter of PPG signals and how to choose the loss functions for deep learning model were also discussed in the study.Finally,the stability of the proposed model was tested using a 10-fold cross-validation,with an MAE±SD of 2.942±5.076 mmHg for SBP,1.747±3.042 mmHg for DBP,and 1.137±2.463 bpm for the HR.Compared with its existing counterparts,the model entailed less computational load and was more accurate in estimating SBP,DBP,and HR.These results established the validity of the model.

A Deep Learning-Based Continuous Blood Pressure Measurement by Dual Photoplethysmography Signals

This study proposed a measurement platform for continuous blood pressure estimation based on dual photoplethysmography(PPG)sensors and a deep learning(DL)that can be used for continuous and rapid measurement of blood pressure and analysis of cardiovascular-related indicators.The proposed platform measured the signal changes in PPG and converted them into physiological indicators,such as pulse transit time(PTT),pulse wave velocity(PWV),perfusion index(PI)and heart rate(HR);these indicators were then fed into the DL to calculate blood pressure.The hardware of the experiment comprised 2 PPG components(i.e.,Raspberry Pi 3 Model B and analog-todigital converter[MCP3008]),which were connected using a serial peripheral interface.The DL algorithm converted the stable dual PPG signals acquired from the strictly standardized experimental process into various physiological indicators as input parameters and finally obtained the systolic blood pressure(SBP),diastolic blood pressure(DBP)and mean arterial pressure(MAP).To increase the robustness of the DL model,this study input data of 100 Asian participants into the training database,including those with and without cardiovascular disease,each with a proportion of approximately 50%.The experimental results revealed that the mean absolute error and standard deviation of SBP was 0.17±0.46 mmHg.The mean absolute error and standard deviation of DBP was 0.27±0.52 mmHg.The mean absolute error and standard deviation of MAP was 0.16±0.40 mmHg.

An Improved Approach to the Performance of Remote Photoplethysmography

Heart rate is an important metric for determining physical and mental health.In recent years,remote photoplethysmography(rPPG)has been widely used in characterizing physiological signals in human subjects.Currently,research on non-contact detection of heart rate mainly focuses on the capture and separation of spectral signals from video imagery.However,this method is very sensitive to the movement of the test subject and light intensity variation,and this results in motion artifacts which presents challenges in extracting accurate physiological signals such as heart rate.In this paper,an improved method for rPPG signal preprocessing is proposed.Based on the well known red green blue(RGB)color space,we segmented skin tone in different color spaces and extracted rPPG signals,after which we use a skin segmentation training model based on the luminance component,the blue-difference chroma components,and red-difference chroma components(YCbCr),as well as hue saturation intensity(HSI)color models.In the experimental verification section,we compare the robustness of the signal on different color spaces.In summary,we are experimentally verifying a better image pre-processing method based on real-time rPPG,which results in more precise measurements through the comparative analysis of skin segmentation and signal quality.

Pulse rate estimation based on facial videos:an evaluation and optimization of the classical methods using both self-constructed and public datasets

Pulse rate is one of the important characteristics of traditional Chinese medicine pulse diagnosis,and it is of great significance for determining the nature of cold and heat in diseases.The prediction of pulse rate based on facial video is an exciting research field for getting palpation information by observation diagnosis.However,most studies focus on optimizing the algorithm based on a small sample of participants without systematically investigating multiple influencing factors.A total of 209 participants and 2,435 facial videos,based on our self-constructed Multi-Scene Sign Dataset and the public datasets,were used to perform a multi-level and multi-factor comprehensive comparison.The effects of different datasets,blood volume pulse signal extraction algorithms,region of interests,time windows,color spaces,pulse rate calculation methods,and video recording scenes were analyzed.Furthermore,we proposed a blood volume pulse signal quality optimization strategy based on the inverse Fourier transform and an improvement strategy for pulse rate estimation based on signal-to-noise ratio threshold sliding.We found that the effects of video estimation of pulse rate in the Multi-Scene Sign Dataset and Pulse Rate Detection Dataset were better than in other datasets.Compared with Fast independent component analysis and Single Channel algorithms,chrominance-based method and plane-orthogonal-to-skin algorithms have a more vital anti-interference ability and higher robustness.The performances of the five-organs fusion area and the full-face area were better than that of single sub-regions,and the fewer motion artifacts and better lighting can improve the precision of pulse rate estimation.

A Deepfake Detection Algorithm Based on Fourier Transform of Biological Signal

Deepfake-generated fake faces,commonly utilized in identity-related activities such as political propaganda,celebrity impersonations,evidence forgery,and familiar fraud,pose new societal threats.Although current deepfake generators strive for high realism in visual effects,they do not replicate biometric signals indicative of cardiac activity.Addressing this gap,many researchers have developed detection methods focusing on biometric characteristics.These methods utilize classification networks to analyze both temporal and spectral domain features of the remote photoplethysmography(rPPG)signal,resulting in high detection accuracy.However,in the spectral analysis,existing approaches often only consider the power spectral density and neglect the amplitude spectrum—both crucial for assessing cardiac activity.We introduce a novel method that extracts rPPG signals from multiple regions of interest through remote photoplethysmography and processes them using Fast Fourier Transform(FFT).The resultant time-frequency domain signal samples are organized into matrices to create Matrix Visualization Heatmaps(MVHM),which are then utilized to train an image classification network.Additionally,we explored various combinations of time-frequency domain representations of rPPG signals and the impact of attention mechanisms.Our experimental results show that our algorithm achieves a remarkable detection accuracy of 99.22%in identifying fake videos,significantly outperforming mainstream algorithms and demonstrating the effectiveness of Fourier Transform and attention mechanisms in detecting fake faces.

Video-Based Deception Detection with Non-Contact Heart Rate Monitoring and Multi-Modal Feature Selection

Deception detection plays a crucial role in criminal investigation.Videos contain a wealth of information regarding apparent and physiological changes in individuals,and thus can serve as an effective means of deception detection.In this paper,we investigate video-based deception detection considering both apparent visual features such as eye gaze,head pose and facial action unit(AU),and non-contact heart rate detected by remote photoplethysmography(rPPG)technique.Multiple wrapper-based feature selection methods combined with the K-nearest neighbor(KNN)and support vector machine(SVM)classifiers are employed to screen the most effective features for deception detection.We evaluate the performance of the proposed method on both a self-collected physiological-assisted visual deception detection(PV3D)dataset and a public bag-oflies(BOL)dataset.Experimental results demonstrate that the SVM classifier with symbiotic organisms search(SOS)feature selection yields the best overall performance,with an area under the curve(AUC)of 83.27%and accuracy(ACC)of 83.33%for PV3D,and an AUC of 71.18%and ACC of 70.33%for BOL.This demonstrates the stability and effectiveness of the proposed method in video-based deception detection tasks.

Gram Matrix-Based Convolutional Neural Network for Biometric Identification Using Photoplethysmography Signal

As a kind of physical signals that could be easily acquired in daily life,photoplethysmography(PPG)signal becomes a promising solution to biometric identification for daily access management system(AMS).State-of-the-art PPG-based identification systems are susceptible to the form of motions and physical conditions of the subjects.In this work,to exploit the advantage of deep learning,we developed an improved deep convolutional neural network(CNN)architecture by using the Gram matrix(GM)technique to convert time-serial PPG signals to two-dimensional images with a temporal dependency to improve accuracy under different forms of motions.To ensure a fair evaluation,we have adopted cross-validation method and“training and testing”dataset splitting method on the TROIKA dataset collected in ambulatory conditions.As a result,the proposed GM-CNN method achieved accuracy improvement from 69.5%to 92.4%,which is the best result in terms of multi-class classification compared with state-of-the-art models.Based on average five-fold cross-validation,we achieved an accuracy of 99.2%,improved the accuracy by 3.3%compared with the best existing method for the binary-class.

New Approach to Measuring the Ankle and Toe Brachial Indices as New Markers for Early Detection of Lower Extremity Peripheral Artery Disease

Background: Lower extremity Peripheral artery disease (PAD) is caused by atherosclerosis, or Plaque buildup, that reduces the blood flow to the legs and feet. PAD affects approximately 230 million adults worldwide and is associated with an increased risk of coronary heart disease, stroke, and leg amputation. The first-line method for diagnosis of PAD is the Ankle Brachial Index (ABI), which is the ratio of ankle to brachial higher systolic pressure measured in ankles and arms. The Toe Brachial Index (TBI), which is the ratio of the toe systolic pressure to brachial higher systolic pressure measured in both arms, is considered to be an alternative to the ABI in screening for PAD. The ABI and TBI are measured on the right and left side, and the lower of these numbers is the patient’s overall ABI and TBI. Clinical studies and meta-analysis reviews have shown that the conventional ABI measurement, which uses a cuff, and handheld sphygmomanometer and continuous-wave Doppler tracings, provides an acceptable-to-high specificity level but low sensitivity when compared with vascular color Doppler ultrasound, and/or angiography methods. Another study has shown that the TBI measurement has greater sensitivity but lower specificity than the ABI when compared with vascular color Doppler ultrasound diagnostic based on waveforms. The aim of this clinical study was to evaluate the specificity and sensitivity of the VasoPad System comparing its results to the vascular color doppler ultrasound waveforms. Materials and Methods: The VasoPad System is an automated device using the pulse wave method to measure the arms and ankles dorsalis and tibial posterior artery blood pressures, the photoplethysmography second derivative (PTGSD) to estimate the toe systolic pressure, a patented photoplethysmography (PTG) index marker and volume plethysmography via cuffs during deflation. Vascular Color Doppler ultrasound can diagnose stenosis through the direct visualization of atherosclerosis or plaques and through waveform analysis. The vascular color Doppler ultrasound provides 3 waveform types. The type 1, triphasic waveform is normal blood flow and no atherosclerosis or plaque, the type 2, diphasic waveform is seen when there are atherosclerosis plaques, but normal blood flow, and the type 3, monophasic waveform reflects stenosis with diameter reduction > 50%. Results: The sum of the overall ABI and TBI VasoPad values, called Sum of Brachial Indices (SBI), gave a specificity of 88.89% and sensitivity of 100% for detecting vascular color Doppler ultrasound biphasic and monophasic waveforms versus triphasic waveforms with a cutoff ≤ 1.36 (P Conclusion: The VasoPad was useful for detecting PAD, which is fully defined as having vessel stenosis > 50% (Doppler monophasic waveforms) but also early stage of atherosclerosis plaque of the lower extremities (Doppler biphasic waveforms). The VasoPad method provided a remarkable sensitivity of 100% and a specificity level similar to those of the conventional ABI test method compared with the vascular color Doppler ultrasound. In addition to being useful to screen and detect PAD, the VasoPad offers early detection of lower extremity atherosclerosis, with normal blood flow (Doppler biphasic waveforms), which could provide greater treatment options and thus reduce the overall number of lower extremity complications.

基于遥测光电容积脉搏波描记法的心率测量综述

遥测光电容积脉搏波描记法(remote photoplethysmography,rPPG)是一种无需接触即可实现心率等生理信号测量的技术,在重症监护、情绪感知、驾驶员状态评估等医疗和工业领域中都有较大的应用潜力。然而基于遥测光电容积脉搏波描记法的非接触心率测量容易受到光照、运动等多种因素干扰,为提高非接触式生理指标测量的精度,中外学者做了大量的研究工作。系统性地综述了基于rPPG的非接触式心率为代表的生理指标测量研究进展。首先,概述了rPPG技术的背景和原理,而后对比分析了基于rPPG心率测量的主流传统方法,此外对基于深度学习的rPPG心率测量最新研究进展进行了分类阐述,最后讨论了非接触心率测量的前景以及未来研究方向。

MSSTNet:Multi-scale facial videos pulse extraction network based on separable spatiotemporal convolution and dimension separable attention

Background The use of remote photoplethysmography(rPPG)to estimate blood volume pulse in a noncontact manner has been an active research topic in recent years.Existing methods are primarily based on a singlescale region of interest(ROI).However,some noise signals that are not easily separated in a single-scale space can be easily separated in a multi-scale space.Also,existing spatiotemporal networks mainly focus on local spatiotemporal information and do not emphasize temporal information,which is crucial in pulse extraction problems,resulting in insufficient spatiotemporal feature modelling.Methods Here,we propose a multi-scale facial video pulse extraction network based on separable spatiotemporal convolution(SSTC)and dimension separable attention(DSAT).First,to solve the problem of a single-scale ROI,we constructed a multi-scale feature space for initial signal separation.Second,SSTC and DSAT were designed for efficient spatiotemporal correlation modeling,which increased the information interaction between the long-span time and space dimensions;this placed more emphasis on temporal features.Results The signal-to-noise ratio(SNR)of the proposed network reached 9.58dB on the PURE dataset and 6.77dB on the UBFC-rPPG dataset,outperforming state-of-the-art algorithms.Conclusions The results showed that fusing multi-scale signals yielded better results than methods based on only single-scale signals.The proposed SSTC and dimension-separable attention mechanism will contribute to more accurate pulse signal extraction.

Power spectral density-based nearinfrared sub-band detection for noninvasive blood glucose prediction in both in-vitro and in-vivo studies

Diabetes is a widespread and serious disease and noninvasive measurement has been in high demand.To address this problem,a power spectral density-based method was offered for determining glucose sensitive sub-bands in the nearinfrared(NIR)spectrum.The experiments were conducted using phantoms of different optical properties in-vitro conditions.The optical bands 1200–1300 nm and 2100–2200 nm were found feasible for measuring blood glucose.After that,a photoplethysmography(PPG)-based low cost and portable optical system was designed.It has six di®erent NIR wavelength LEDs for illumination and an InGaAs photodiode for detection.Optical density values were calculated through the system and used as independent variables for multiple linear regression analysis.The results of blood glucose levels for 24 known healthy subjects showed that the optical system prediction was nearly 80%in the A zone and 20%in the B zone according to the Clarke Error Grid analysis.It was shown that a promising easyuse,continuous,and compact optical system had been designed.

NONINVASIVE PROBING OF THE NEUROVASCULAR SYSTEM IN HUMAN BONE/BONE MARROW USING NEAR-INFRARED LIGHT

Understanding the mechanisms of interaction between bone/bone marrow,circulatory system and nervous system is of great interest due to the potential clinical impact.In humans,the amount of knowledge in this domain remains relatively limited due to the extreme difficulty to monitor these tissues continuously,noninvasively and for long or repeated periods of time.A typical difficult task would be,for example,to continuously monitor bone/bone marrow blood perfusion,hemoglobin oxygen saturation or blood volume and study their dependence on the activity of the autonomic nervous system.In this review article,we want to show that nearinfrared light might be utilized to solve these problems in part.We hope that the present analysis will stimulate future studies in this domain,for which near-infrared light appears as the best available technology today.

Bone blood flow is influenced by muscle contractions

Forces acting on the skeleton could be divided into those originating from gravitational loading and those originating from muscle loading. Flat bones in a non-weight-baring segment of the skeleton probably experience forces mostly generated by muscle contractions. One purpose of muscle contractions is to generate blood flow within skeletal tissues. The present study aimed to investigate the pulsatile patellar bone blood flow after low and high intensity leg extension exercises. Forty-two healthy individuals volunteered for the study. Dynamic isotonic one leg extension/flexion exercises were performed in a leg extension machine. Randomly, the exercises were performed with the left or right leg with either 10 repetition maximum (10 RM) continuously without any resting periods (high intensity muscle work), or 20 RM with a 2 second rest between contractions (low intensity muscle work). The work load, expressed in kilograms totally lifted, was identical in both legs. The pulsatile patellar blood flow was recorded continuously using a photoplethysmographic technique. Blood pressure was measured continuously during muscle work by a non-invasive method (Finapress). The patellar pulsatile bone blood flow increased significantly more after high intensity muscle work (61%) compared to the same work load performed using a lower intensity (22%), p = 0.000073. Systolic blood pressure changed equally during and after both interventions. Post-exercise bone hyperaemia appears to be correlated to the intensity of muscle contractions in the muscle compartment attached to the bone.

Dual-domain and Multiscale Fusion Deep Neural Network for PPG Biometric Recognition

Photoplethysmography(PPG)biometrics have received considerable attention.Although deep learning has achieved good performance for PPG biometrics,several challenges remain open:1)How to effectively extract the feature fusion representation from time and frequency PPG signals.2)How to effectively capture a series of PPG signal transition information.3)How to extract timevarying information from one-dimensional time-frequency sequential data.To address these challenges,we propose a dual-domain and multiscale fusion deep neural network(DMFDNN)for PPG biometric recognition.The DMFDNN is mainly composed of a two-branch deep learning framework for PPG biometrics,which can learn the time-varying and multiscale discriminative features from the time and frequency domains.Meanwhile,we design a multiscale extraction module to capture transition information,which consists of multiple convolution layers with different receptive fields for capturing multiscale transition information.In addition,the dual-domain attention module is proposed to strengthen the domain of greater contributions from time-domain and frequency-domain data for PPG biometrics.Experiments on the four datasets demonstrate that DMFDNN outperforms the state-of-the-art methods for PPG biometrics.

Advances in Polymethine Dyes for Near-Infrared Organic Photodiodes

Near-infrared organic photodiodes (NIR OPDs) have tremendous potential in industrial, military, and scientific applications, due to their unique features of lightweight, low toxicity, high structural flexibility, cooling-system-free, etc. However, the overall performance of currently available NIR OPDs still lags behind the commercial inorganic photodetectors, ascribed to the critical challenge of realizing organic semiconductors with sufficiently low optical bandgap and excellent optoelectronic properties simultaneously. Among various types of NIR-absorbing organic semiconductors, polymethine dyes not only possess advantages of simple synthesis and structural diversity, but also show fascinating optical and aggregation features in the solid state, making them attractive material candidates for NIR OPDs. In this review, after a brief introduction of NIR OPDs and polymethine dyes, we comprehensively summarize the advances of polymethine dyes for broadband and narrowband NIR OPDs, and further introduce their applications in all-organic optical upconversion devices and photoplethysmography sensors. In particular, the relationship between the chemical structure and the aggregation behaviors of polymethine dyes and the device performance is carefully discussed, providing some important molecular insights for developing high performance NIR OPDs.

Remote heart rate measurement using low-cost RGB face video： a technical literature review

Remote photoplethysmography （rPPG） allows remote measurement of the heart rate using low-cost RGB imaging equipment. In this study, we review the development of the field of rPPG since its emergence in 2008. We also classify existing rPPG approaches and derive a framework that provides an overview of modular steps. Based on this framework, practitioners can use our classification to design algorithms for an rPPG approach that suits their specific needs. Researchers can use the reviewed and classified algorithms as a starting point to improve particular features of an rPPG algorithm.

Organic photodiodes:device engineering and applications

Organic photodiodes(OPDs)have shown great promise for potential applications in optical imaging,sensing,and communication due to their wide-range tunable photoelectrical properties,low-temperature facile processes,and excellent mechanical fexibility.Extensive research work has been carried out on exploring materials,device structures,physical mechanisms,and processing approaches to improve the performance of OPDs to the level of their inorganic counterparts.In addition,various system prototypes have been built based on the exhibited and attractive features of OPDs.It is vital to link the device optimal design and engineering to the system requirements and examine the existing defciencies of OPDs towards practical applications,so this review starts from discussions on the required key performance metrics for diferent envisioned applications.Then the fundamentals of the OPD device structures and operation mechanisms are briefy introduced,and the latest development of OPDs for improving the key performance merits is reviewed.Finally,the trials of OPDs for various applications including wearable medical diagnostics,optical imagers,spectrometers,and light communications are reviewed,and both the promises and challenges are revealed.

In vivo skin imaging prototypes ＂made in Latvia＂

This paper briefly reviews the operational principles and designs of portable in vivo skin imaging prototypes developed at the Biophotonics Laboratory of the Institute of Atomic Physics and Spectroscopy, University of Latvia. Four types of imaging devices are presented. Multi-spectral imagers ensure distant mapping of specific skin parameters （e.g., distribution of skin chromophores）. Autofluorescence photobleaching rate imagers show potential for skin tumor assessment and margin delineation. Photoplethysmography video-imagers remotely detect cutaneous blood pulsations and provide real-time information on the human cardiovascular state. Multimodal skin imagers perform the above-mentioned functions by acquiring several spectral and video images using the same image sensor.