Respiration Based Non-Invasive Approach for Emotion Recognition Using Impulse Radio Ultra Wide Band Radar and Machine Learning
Journal article
Siddiqui, H., Shahzad, H., Saleem, A., Khan Khakwani, Abdul Baqi, Rustam, F., Lee, E., Ashraf, I. and Dudley-Mcevoy, S. (2021). Respiration Based Non-Invasive Approach for Emotion Recognition Using Impulse Radio Ultra Wide Band Radar and Machine Learning. Sensors. 21 (24), p. e8336. https://doi.org/10.3390/s21248336
| Authors | Siddiqui, H., Shahzad, H., Saleem, A., Khan Khakwani, Abdul Baqi, Rustam, F., Lee, E., Ashraf, I. and Dudley-Mcevoy, S. |
|---|---|
| Abstract | Emotion recognition gained increasingly prominent attraction from a multitude of fields recently due to their wide use in human-computer interaction interface, therapy, and advanced robotics, etc. Human speech, gestures, facial expressions, and physiological signals can be used to recognize different emotions. Despite the discriminating properties to recognize emotions, the first three methods have been regarded as ineffective as the probability of human’s voluntary and involuntary concealing the real emotions can not be ignored. Physiological signals, on the other hand, are capable of providing more objective, and reliable emotion recognition. Based on physiological signals, several methods have been introduced for emotion recognition, yet, predominantly such approaches are invasive involving the placement of on-body sensors. The efficacy and accuracy of these approaches are hindered by the sensor malfunctioning and erroneous data due to human limbs movement. This study presents a non-invasive approach where machine learning complements the impulse radio ultra-wideband (IR-UWB) signals for emotion recognition. First, the feasibility of using IR-UWB for emotion recognition is analyzed followed by determining the state of emotions into happiness, disgust, and fear. These emotions are triggered using carefully selected video clips to human subjects involving both males and females. The convincing evidence that different breathing patterns are linked with different emotions has been leveraged to discriminate between different emotions. Chest movement of thirty-five subjects is obtained using IR-UWB radar while watching the video clips in solitude. Extensive signal processing is applied to the obtained chest movement signals to estimate respiration rate per minute (RPM). The RPM estimated by the algorithm is validated by repeated measurements by a commercially available Pulse Oximeter. A dataset is maintained comprising gender, RPM, age, and associated emotions which are further used with several machine learning algorithms for automatic recognition of human emotions. Experiments reveal that IR-UWB possesses the potential to differentiate between different human emotions with a decent accuracy of 76% without placing any on-body sensors. Separate analysis for male and female participants reveals that males experience high arousal for happiness while females experience intense fear emotions. For disgust emotion, no large difference is found for male and female participants. To the best of the authors’ knowledge, this study presents the first non-invasive approach using the IR-UWB radar for emotion recognition. |
| Keywords | machine learning; non-invasive emotion recognition; physiological signals; respiration rate; ultra-wide band |
| Year | 2021 |
| Journal | Sensors |
| Journal citation | 21 (24), p. e8336 |
| Publisher | MDPI |
| ISSN | 1424-8220 |
| Digital Object Identifier (DOI) | https://doi.org/10.3390/s21248336 |
| Funder/Client | Florida Center for Advanced Analytics and Data Science funded by Ernesto.Net (under the Algorithms for Good Grant) |
| Publication dates | |
| Online | 13 Dec 2021 |
| Publication process dates | |
| Deposited | 11 Jan 2022 |
| Accepted | 09 Dec 2021 |
| Publisher's version | File Access Level Open |
| License | https://creativecommons.org/licenses/by/4.0/ |
Permalink -
https://openresearch.lsbu.ac.uk/item/8z094
Download files
145
total views146
total downloads0
views this month2
downloads this month
Export as
Related outputs
Automated Marker-based Abnormal Gait Pattern Detection Using Novel 6-dimensional Skeleton
Wickramarachchi, W., Brown, R., Ghavami, M., Berthaume, M. and Dudley-Mcevoy, S. (2024). Automated Marker-based Abnormal Gait Pattern Detection Using Novel 6-dimensional Skeleton. ISMICT 2024. London, UK 15 - 17 May 2024 Institute of Electrical and Electronics Engineers (IEEE).Huygens Principle based Microwave Imaging for Lung Lesion Detection using Realistic Phantom
Khalid, B., Khalesi, B., Ghavami, N., Brown, R., Dudley-Mcevoy, S., Ghavami, M. and Tiberi, G. (2024). Huygens Principle based Microwave Imaging for Lung Lesion Detection using Realistic Phantom. ISMICT 2024. London, UK 15 - 17 May 2024 Institute of Electrical and Electronics Engineers (IEEE).Ultra-Wide Band Radar Empowered Driver Drowsiness Detection with Convolutional Spatial Feature Engineering and Artificial Intelligence
Siddiqui, H., Akmal, A., Iqbal, M., Saleem, A., Raza, M., Zafar, K., Zaib, A., Dudley-Mcevoy, S., Arambarri, J., Castilla, Á. and Rustam, F. (2024). Ultra-Wide Band Radar Empowered Driver Drowsiness Detection with Convolutional Spatial Feature Engineering and Artificial Intelligence. Sensors. 24 (12), p. 3754. https://doi.org/10.3390/s24123754Modified Flower Pollination Algorithm for Energy Forecasting and Demand Management Coupled with Improved Battery Life for Smart Building Micro-Grid
Hamza, Ali, Ali, Zunaib, Dudley, Sandra, Uneeb, Muhammad, Alghamdi, Sultan M and Christofides, Nicholas (2024). Modified Flower Pollination Algorithm for Energy Forecasting and Demand Management Coupled with Improved Battery Life for Smart Building Micro-Grid. 2024 IEEE Texas Power and Energy Conference (TPEC). https://doi.org/10.1109/tpec60005.2024.10472245Gender classification based on gait analysis using ultrawide band radar augmented with artificial intelligence
Dudley-Mcevoy, S., Saleem, A., Siddiqui, H. and Sehar, R. (2024). Gender classification based on gait analysis using ultrawide band radar augmented with artificial intelligence. Expert Systems with Applications. 249 (PART C). https://doi.org/http10.1016/j.eswa.2024.123843Power Grid Frequency Forecasting from μPMU Data using Hybrid Vector-Output LSTM network
Dey, M., Wickramarachchi, D., Rana, S.P., Simmons, .C.V and Dudley, S. (2023). Power Grid Frequency Forecasting from μPMU Data using Hybrid Vector-Output LSTM network. 2023 IEEE PES Innovative Smart Grid Technologies Europe (ISGT EUROPE). 23 - 26 Oct 2023 IEEE. https://doi.org/10.1109/ISGTEUROPE56780.2023.10408056Distribution Substation Dynamic Reconfiguration and Reinforcement-Digital Twin Model
Brown, R., Wickramarachchi, W., Ali, Z., Saleem, K. and Dudley-Mcevoy, S. (2023). Distribution Substation Dynamic Reconfiguration and Reinforcement-Digital Twin Model. IEEE Energy Conversion Congress and Exposition (ECCE). Center in Nashville, Tennessee | October 29 – November 2, 2023 29 Oct - 02 Nov 2023 Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/ECCE53617.2023.10362160Power Grid Frequency Forecasting from μPMU Data using Hybrid Vector-Output LSTM network
Dey, M., Wickramarachchi, D., Rana, S.P., Simmons, C.v. and Dudley, S. (2023). Power Grid Frequency Forecasting from μPMU Data using Hybrid Vector-Output LSTM network. 2023 IEEE PES Innovative Smart Grid Technologies Europe (ISGT EUROPE). https://doi.org/10.1109/isgteurope56780.2023.10408056Novel Parameter-Free and Parametric Same Degree Distribution-based Dimensionality Reduction Algorithms for Trustworthy Data Structure Preserving
Hajderanj, L., Chen, D. and Dudley-Mcevoy, S. (2023). Novel Parameter-Free and Parametric Same Degree Distribution-based Dimensionality Reduction Algorithms for Trustworthy Data Structure Preserving. Information Sciences. 661, p. 120030. https://doi.org/10.1016/j.ins.2023.120030Cost-effective and Resilient Operation of Distribution Grids and 5G Telecommunication
Wang, J., Qiu, D., Wang, Y., Ghosh, S., Pinson, P., Dudley, S. and Strbac, G. (2023). Cost-effective and Resilient Operation of Distribution Grids and 5G Telecommunication. 2023 IEEE Power & Energy Society General Meeting (PESGM). https://doi.org/10.1109/pesgm52003.2023.10252696Footwear-integrated force sensing resistor sensors: A machine learning approach for categorizing lower limb disorders
Siddiqui, H.U.R., Nawaz, S., Saeed, M., Saleem, A., Raza, M.A., Raza, A., Aslam, M.A. and Dudley, S. (2023). Footwear-integrated force sensing resistor sensors: A machine learning approach for categorizing lower limb disorders. Engineering Applications of Artificial Intelligence. 127, p. 107205. https://doi.org/10.1016/j.engappai.2023.107205Empowering Lower Limb Disorder Identification through PoseNet and Artificial Intelligence
Siddiqui, H., Saleem, A., Raza, M., Villar, S., Lopez, L., Díez, I., Rustam, F. and Dudley-Mcevoy, S. (2023). Empowering Lower Limb Disorder Identification through PoseNet and Artificial Intelligence. Diagnostics. 13 (18), p. 2881. https://doi.org/10.3390/diagnostics13182881Huygens Principle-based Microwave Brain Imaging through Finite Difference Time Domain
Movafagh, M., Ghavami, N., Tiberi, G., Dudley-Mcevoy, S. and Ghavami, M. (2023). Huygens Principle-based Microwave Brain Imaging through Finite Difference Time Domain. 2023 IEEE Conference on Antenna Measurements & Applications. Genoa, Italy 15 - 17 Nov 2023 Institute of Electrical and Electronics Engineers (IEEE).Enhancing Cricket Performance Analysis with Human Pose Estimation and Machine Learning
Siddiqui, H., Faizan Y., Rustam, F., Soriano, E., Ballester, J.B., Díez, I., Dudley-Mcevoy, S. and Ashraf, I. (2023). Enhancing Cricket Performance Analysis with Human Pose Estimation and Machine Learning. Sensors. 23 (15), p. 6839. https://doi.org/10.3390/s23156839Cutting-edge strategies for breast lesions detection through Radiomics in novel microwave imaging technologies: Features' extraction reliability on microwave images from MammoWave device
Tiberi, G., Ghavami, M., Dudley-Mcevoy, S., Ghavami, N. and Alvarez Sánchez-Bayuela, D. (2023). Cutting-edge strategies for breast lesions detection through Radiomics in novel microwave imaging technologies: Features' extraction reliability on microwave images from MammoWave device. ECR 2023. Vienna 01 - 05 Mar 2023 European Society of Radiology. https://doi.org//10.26044/ecr2023/C-18521A Spiral-like Acquisition Strategy for 3D Huygens' Principle Based Microwave Imaging
Khalid, B., Khalesi, B., Ghavami, N., Raspa, G., Badia, M., Dudley-Mcevoy, S., Ghavami, M. and Tiberi, G. (2023). A Spiral-like Acquisition Strategy for 3D Huygens' Principle Based Microwave Imaging. Photonics & Electromagnetics Research Symposium (Piers) 2023. Prague 06 Jun - 03 Jul 2023 Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/PIERS59004.2023.10221326An Approach to Detect Chronic Obstructive Pulmonary Disease Using UWB Radar-Based Temporal and Spectral Features
Siddiqui, H., Ali Raza, Saleem, A., Rustam, F., Isabel De la Torre, Daniel Gavilanes Aray, Vivian Lipari, Ashraf, I. and Dudley-Mcevoy, S. (2023). An Approach to Detect Chronic Obstructive Pulmonary Disease Using UWB Radar-Based Temporal and Spectral Features. Diagnostics. 13 (6), p. 1096. https://doi.org/10.3390/diagnostics13061096Data Driven Machine Learning Model for Condition Monitoring and Anomaly Detection in Power Grids
Saleem, K., Alkan, B. and Dudley-Mcevoy, S. (2023). Data Driven Machine Learning Model for Condition Monitoring and Anomaly Detection in Power Grids. 2023 IEEE Power & Energy Society General Meeting. Orlando, Florida, US 10 - 16 Jul 2023 Institute of Electrical and Electronics Engineers (IEEE).Radiation-free Microwave Technology for Breast Lesion Detection using Supervised Machine Learning Model
Rana, S., Dey, M., Loretoni, R., Duranti, M., Ghavami, M., Dudley-Mcevoy, S. and Tiberi, G. (2023). Radiation-free Microwave Technology for Breast Lesion Detection using Supervised Machine Learning Model. Tomography. 9 (1), pp. 105-129. https://doi.org/10.3390/tomography9010010A systematic review of physiological signals based driver drowsiness detection systems.
Saleem, A.., Siddiqui, H.U.R., Raza, M.A., Rustam, F., Dudley-Mcevoy, S. and Ashraf, I. (2022). A systematic review of physiological signals based driver drowsiness detection systems. Cognitive neurodynamics. 17 (5), pp. 1229-1259. https://doi.org/10.1007/s11571-022-09898-9Microgrid Cyberphysical Systems ; Chapter 5 - Control of PV and EV connected smart grid
Ali, Z., Saleem, K., Putrus, G., Marzband, M. and Dudley-Mcevoy, S. (2022). Microgrid Cyberphysical Systems ; Chapter 5 - Control of PV and EV connected smart grid. in: Subudhi, B. (ed.) Elsevier Renewable Energy and Plug-In Vehicle Integration Elsevier Renewable Energy and Plug-In Vehicle Integration.Detecting Power Grid Frequency Events from μPMU Voltage Phasor Data Using Machine Learning
Dey, M., Rana, S., Wylie, J., Simmons, C. V. and Dudley-Mcevoy, S. (2022). Detecting Power Grid Frequency Events from μPMU Voltage Phasor Data Using Machine Learning. The IET 11th International Conference on Renewable Power Generation. IET London: Savoy Place. 22 - 23 Sep 2022 Institute of Engineering and Technology (IET).Automatic User Preferences Selection of Smart Hearing Aid Using BioAid
Siddiqui, H., Saleem, A., Raza, M., Kainat Zafar, Riccardo Russo and Dudley-Mcevoy, S. (2022). Automatic User Preferences Selection of Smart Hearing Aid Using BioAid. Sensors. 22 (20), p. 8031. https://doi.org/10.3390/s222080313D Huygens Principle based Microwave Imaging through MammoWave Device: Validation through Phantoms.
Khalid, B., Khalesi, B., Ghavami, N., Sani, L., Vispa, A., Madia, M., Dudley-Mcevoy, S., Ghavami, M. and Tiberi, G. (2022). 3D Huygens Principle based Microwave Imaging through MammoWave Device: Validation through Phantoms. IEEE Access. Volume (10), pp. 106770 - 106780. https://doi.org/10.1109/ACCESS.2022.3211957Respiration-Based COPD Detection Using UWB Radar Incorporation with Machine Learning
Siddiqui, H., Saleem, A., Imran B., Kainat Z., Rustam, F., De la Torre, I., Dudley-Mcevoy, S. and Ashraf, I. (2022). Respiration-Based COPD Detection Using UWB Radar Incorporation with Machine Learning. Electronics. 11 (18), p. 2875. https://doi.org/10.3390/electronics11182875Automated breast lesion localisation in microwave imaging employing simplified pulse coupled neural network
Dey, M., Rana, S., Loretoni, R., Duranti, M., Sani, L., Vispa, A., Raspa, G., Ghavami, M., Dudley-Mcevoy, S. and Tiberi, G. (2022). Automated breast lesion localisation in microwave imaging employing simplified pulse coupled neural network. PLoS ONE. https://doi.org/10.1371/journal.pone.0271377Performance Analysis and Benchmarking of PLL-Driven Phasor Measurement Units for Renewable Energy Systems
Ali, Z., Saleem, K., Brown, R., Christofides, N. and Dudley-Mcevoy, S. (2022). Performance Analysis and Benchmarking of PLL-Driven Phasor Measurement Units for Renewable Energy Systems. Energies. 15 (5), p. e1867. https://doi.org/10.3390/en15051867Markerless Gait Classification Employing 3D IR-UWB Physiological Motion Sensing
Rana, S., Dey, M., Ghavami, M. and Dudley-Mcevoy, S. (2022). Markerless Gait Classification Employing 3D IR-UWB Physiological Motion Sensing. IEEE Sensors Journal. 22 (7), pp. 6931-6941. https://doi.org/10.1109/JSEN.2022.3154092An Analytically Based Approach for Evaluating the Impact of the Noise on the Microwave Imaging Detection
Sohani, Behnaz, Abdallah, Adam Degaichia, Tiberi, Gianluigi, Ghavami, Navid, Ghavami, Mohammad and Dudley, Sandra (2021). An Analytically Based Approach for Evaluating the Impact of the Noise on the Microwave Imaging Detection. 2021 Photonics & Electromagnetics Research Symposium (PIERS). https://doi.org/10.1109/piers53385.2021.9695034Radial Basis Function for Breast Lesion Detection from MammoWave Clinical Data
Rana, S., Dey, M., Riccardo Loretoni, Michele Duranti, Lorenzo Sani, Alessandro Vispa, Ghavami, M., Sandra Dudley and Gianluigi Tiberi (2021). Radial Basis Function for Breast Lesion Detection from MammoWave Clinical Data. Diagnostics. 11 (10). https://doi.org/10.3390/diagnostics11101930A Novel Approach to Railway Track Faults Detection Using Acoustic Analysis.
Shafique, R., Siddiqui, H., Rustam, F., Ullah, S., Siddique, Muhammad Abubakar, Lee, E., Ashraf, I. and Dudley-Mcevoy, S. (2021). A Novel Approach to Railway Track Faults Detection Using Acoustic Analysis. Sensors. 21 (18). https://doi.org/s21186221Solar farm voltage anomaly detection using high-resolution μ PMU data-driven unsupervised machine learning
Dey, M., Rana, S., Simmons, Clarke V. and Dudley-Mcevoy, S. (2021). Solar farm voltage anomaly detection using high-resolution μ PMU data-driven unsupervised machine learning. Applied Energy. 303, p. 117656. https://doi.org/10.1016/j.apenergy.2021.1176563D Microwave Imaging Using Huygens Principle: A Phantom-based Validation
Khalid, B., Khalesi, B., Ghavami, N., Tiberi, G., Dudley-Mcevoy, S. and Ghavami, M. (2021). 3D Microwave Imaging Using Huygens Principle: A Phantom-based Validation. Progress In Electromagnetics Research Symposium (PIERS 2021). Hangzhu 21 - 25 Nov 2021 Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/PIERS53385.2021.96950903D Microwave Imaging Using Huygens Principle: A Phantom-based Validation
Khalid, B., Khalesi, B., Ghavami, N., Dudley, S., Ghavami, M. and Tiberi, G. (2021). 3D Microwave Imaging Using Huygens Principle: A Phantom-based Validation. 2021 Photonics & Electromagnetics Research Symposium (PIERS). https://doi.org/10.1109/piers53385.2021.9695090Microwave Imaging for Lung Covid-19 Infection Detection through Huygens Principle
Ghavami, M., Khalesi, B., Khalid, B., Ghavami, N., Dudley-Mcevoy, S. and Tiberi, G. (2021). Microwave Imaging for Lung Covid-19 Infection Detection through Huygens Principle. Progress In Electromagnetics Research Symposium (PIERS 2021). Hangzhu 21 - 25 Nov 2021