Recurrent Neural Networks for Decoding Lip Read Speech

Conference paper

Fenghour, S, Chen, D and Xiao, P (2019). Recurrent Neural Networks for Decoding Lip Read Speech. 2019 8th International Conference on Software and Information Engineering (ICSIE 2019). Cairo 09 - 12 Apr 2019
AuthorsFenghour, S, Chen, D and Xiao, P
TypeConference paper
Abstract

The success of automated lip reading has been constrained by the inability to distinguish between homopheme words, which are words have different characters and produce the same lip movements (e.g. ”time” and ”some”), despite being intrinsically different. One word can often have different phonemes (units of sound) producing exactly the viseme or visual equivalent of phoneme for a unit of sound. Through the use of a Long-Short Term Memory Network with word embeddings, we can distinguish between homopheme words or words that produce identical lip movements. The neural network architecture achieved a character accuracy rate of 77.1% and a word accuracy rate of 72.2%.

Year2019
Accepted author manuscript
License
CC BY 4.0
Publication dates
Print09 Apr 2019
Publication process dates
Deposited20 Mar 2019
Accepted18 Mar 2019
Permalink -

https://openresearch.lsbu.ac.uk/item/866z8

  • 25
    total views
  • 17
    total downloads
  • 8
    views this month
  • 7
    downloads this month

Related outputs

Distributed deep networks based on Bagging-Down SGD algorithm
Qin, C, Gao, X and Chen, D (2019). Distributed deep networks based on Bagging-Down SGD algorithm. Xi Tong Gong Cheng Yu Dian Zi Ji Shu/Systems Engineering and Electronics. 41 (5), pp. 1021-1027.
Design of a voice control 6DoF grasping robotic arm based on ultrasonic sensor, computer vision and Alexa voice assistance
Wang, Z, Chen, D and Xiao, P (2019). Design of a voice control 6DoF grasping robotic arm based on ultrasonic sensor, computer vision and Alexa voice assistance. International Conference on Information Technology in Medicine and Education. Qingdao, China 23 - 25 Aug 2019
Towards automated cost analysis, benchmarking and estimating in construction: a machine learning approach
Chen, D, Hajderanj, L and Fiske, J (2019). Towards automated cost analysis, benchmarking and estimating in construction: a machine learning approach. 13th Multi Conference on Computer Science and Information Systems (MCCSIS). Porto, Portugal 16 - 18 Jul 2019
In-Vivo Skin Capacitive Image Classification Using AlexNet Convolution Neural Network
Zhang, X, Pan, W and Xiao, P (2018). In-Vivo Skin Capacitive Image Classification Using AlexNet Convolution Neural Network. 2018 3rd International Conference on Image, Vision and Computing (ICIVC 2018). Chongqing, China 27 - 29 Jun 2018 pp. 439-443 doi:10.1109/ICIVC.2018.8492860
Educational Network Bandwidth Analysis and Prediction
Oumar, O A, Dyllon, S and Xiao, P (2018). Educational Network Bandwidth Analysis and Prediction. the 14th International Conference on Machine Learning and Data Mining (MLDM'2018) July 14-19, 2018. New York, USA 14 - 19 Jul 2018
Educational Bandwidth Traffic Prediction using Non-Linear Autoregressive Neural Networks
Oumar, O A, Dyllon, S, Xiao, P and Hong, T (2018). Educational Bandwidth Traffic Prediction using Non-Linear Autoregressive Neural Networks. The 21st International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines - CLAWAR 2018. Panama 10 - 12 Sep 2018
Building an online interactive 3D virtual world for aquaflux and epsilon
Hashimi, OA and Xiao, P (2018). Building an online interactive 3D virtual world for aquaflux and epsilon. Advances in Science, Technology and Engineering Systems. 3 (6), pp. 501-514.
Epsilon Interactive Virtual User Manual (VUM)
Hashimi, OA and Xiao, P (2019). Epsilon Interactive Virtual User Manual (VUM). 2018 International Conference on Computing, Electronics and Communications Engineering, ICCECE 2018. Southend-on-Sea 16 - 17 Aug 2018 pp. 138-143 doi:10.1109/iCCECOME.2018.8658872
Micro-relief analysis with skin capacitive imaging
Bontozoglou, C, Zhang, X and Xiao, P (2019). Micro-relief analysis with skin capacitive imaging. Skin Research and Technology. 25 (2), pp. 165-170.
Improving Prediction Accuracy of Breast Cancer Survivability and Diabetes Diagnosis via RBF Networks trained with EKF models
Adegoke, V, Chen, D and Banissi, E (2019). Improving Prediction Accuracy of Breast Cancer Survivability and Diabetes Diagnosis via RBF Networks trained with EKF models. International Journal of Computer Information Systems and Industrial Management.
On Distributed Deep Network for Processing Large-Scale Sets of Complex Data
Qin, C, Gao, X and Chen, D (2016). On Distributed Deep Network for Processing Large-Scale Sets of Complex Data. 2016 8th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC). Hangzhou, China. 27 - 28 Aug 2016 doi:10.1109/IHMSC.2016.55
In Vivo Human Hair Hydration Measurements by Using Opto-Thermal Radiometry
Bontozoglou, C, Zhang, X, Patel, A, Lane, ME and Xiao, P (2019). In Vivo Human Hair Hydration Measurements by Using Opto-Thermal Radiometry. International Journal of Thermophysics. 40 (2).
Enhancing Ensemble Prediction Accuracy of Breast Cancer Survivability and Diabetes Diagnostic using optimized EKF-RBFN trained prototypes, The 10th International Conference on Soft Computing and Pattern Recognition
Adegoke, V, Chen, D, Banissi, E and Barikzai, S (2019). Enhancing Ensemble Prediction Accuracy of Breast Cancer Survivability and Diabetes Diagnostic using optimized EKF-RBFN trained prototypes, The 10th International Conference on Soft Computing and Pattern Recognition. The 10th International Conference on Soft Computing and Pattern Recognition. Porto, Portugal 13 - 15 Dec 2018
Decoder-Encoder LSTM for Lip Reading
Fanghour, S, Chen, D and Xiao, P (2019). Decoder-Encoder LSTM for Lip Reading. 2019 8th International Conference on Software and Information Engineering (ICSIE 2019). Cairo, Eygpt 09 - 12 Apr 2019
Learning Bayesian network parameters via minimax algorithm
Gao, X, Gao, G, Ren, H, Chen, D and He, C (2019). Learning Bayesian network parameters via minimax algorithm. International Journal of Approximate Reasoning. 108, pp. 62-75.
A New Supervised t-SNE with Dissimilarity Measure for Effective Data Visualization and Classification
Hajderanj, L, Weheliye, I and Chen, D (2019). A New Supervised t-SNE with Dissimilarity Measure for Effective Data Visualization and Classification. 2019 8th International Conference on Software and Information Engineering. Cairo 09 - 12 Apr 2019
Learning Bayesian Networks using the Constrained Maximum a Posteriori Probability Method
Yang, Y, Gao, X, Guo, Z and Chen, D (2019). Learning Bayesian Networks using the Constrained Maximum a Posteriori Probability Method. Pattern Recognition. 91, pp. 123-134.
Contour Mapping for Speaker-Independent Lip Reading System
Fenghour, S, Chen, D and Xiao, P (2018). Contour Mapping for Speaker-Independent Lip Reading System. The 11th International Conference on Machine Vision (ICMV 2018). Munich, Germany 01 - 03 Nov 2018
Predicting Customer Profitability Dynamically over Time: An Experimental Comparative Study
Chen, D., Guo, K. and Li, B. (2019). Predicting Customer Profitability Dynamically over Time: An Experimental Comparative Study. 24th Iberoamerican Congress on Pattern Recognition (CIARP 2019). Havana, Cuba 28 - 31 Oct 2019
Visual analytics in the public sector: An analysis on diversities and similarities of London’s wards
Chen, D, Sanz, BM and Zhao, E (2018). Visual analytics in the public sector: An analysis on diversities and similarities of London’s wards. International Conference on Big Data Analytics, Data Mining and Computational Intelligence 2018 (BigDaCI 2018). Madrid, Spain 18 - 20 Jul 2018 Bigdaci.
Developing a web based interactive 3D virtual environment for novel skin measurement instruments
Xiao, P and Al Hashimi, O (2018). Developing a web based interactive 3D virtual environment for novel skin measurement instruments. 2018 Advances in Science and Engineering Technology International Conferences (ASET). Dubai, Sharjah, Abu Dhabi, United Arab Emirates 06 - 07 Feb 2018 London South Bank University. doi:10.1109/ICASET.2018.8376823
Making Better Use of Big Data
Chen, D (2016). Making Better Use of Big Data. LSBU Enterprise Count Event, March 2016. London Southbank University 18 - 18 Mar 2016 London South Bank University.
Determining Key (Predictor) Modules for Early Identification of Students At-Risk
Chen, D and Elliott, G (2013). Determining Key (Predictor) Modules for Early Identification of Students At-Risk. International Conference on Advanced Information Engineering and Education Science (ICAIEES 2013). Beijing, China 19 - 20 Dec 2013 London South Bank University. doi:10.2991/icaiees-13.2013.22
Big Data Analytics System for Fact/Data-driven Decision Making
Chen, D (2015). Big Data Analytics System for Fact/Data-driven Decision Making. The Royal Statistical Society, Business and Industry Section. London, UK 18 Nov 2015 London South Bank University.
A Bayesian Approach to Learn Bayesian Networks Using Data and Constraints
Gao, X, Yu, Y, Zhi-gao, G and Chen, D (2016). A Bayesian Approach to Learn Bayesian Networks Using Data and Constraints. 23rd International Conference on Pattern Recognition (ICPR 2016). Cancún, México 04 - 08 Dec 2016 London South Bank University. doi:10.1109/ICPR.2016.7900204
Data mining for the online retail industry: A case study of RFM model-based customer segmentation using data mining
Chen, D (2012). Data mining for the online retail industry: A case study of RFM model-based customer segmentation using data mining. Journal of Database Marketing and Customer Strategy Management. 19 (3), pp. 197-208.
Predictive Ensemble Modelling: An Experimental Comparison of Boosting Implementation Methods
Adegoke, V, Chen, D, Barikzai, S and Banissi, E (2017). Predictive Ensemble Modelling: An Experimental Comparison of Boosting Implementation Methods. 2017 European Modelling Symposium (EMS). Manchester 20 - 21 Nov 2017 London South Bank University.
Prediction of Breast Cancer Survivability using Ensemble Algorithms
Adegoke, V, Chen, D, Banissi, E and Barikzai, S (2017). Prediction of Breast Cancer Survivability using Ensemble Algorithms. International Conference on Smart System and Technologies 2017 (SST 2017),. Osijek, Croatia 18 - 20 Oct 2017 London South Bank University.
Learning Bayesian Network Parameters from a Small Data Set: A Further Constrained Qualitatively Maximum a Posteriori Method
Guo, Zhi-gao, Gao, Xiao-guang, Hao, Ren, Yang, Yu, Di, Ruo-hai and Chen, D (2017). Learning Bayesian Network Parameters from a Small Data Set: A Further Constrained Qualitatively Maximum a Posteriori Method. International Journal of Approximate Reasoning. 91 (Dec), pp. 22-35.
Feature Extraction and Labelling Large Data Sets Using Deep Learning
Chen, D (2017). Feature Extraction and Labelling Large Data Sets Using Deep Learning. RESEARCHER LINK: Smart Technology for Fighting Virus Epidemics & Bioinformatics. Recife, Pernambuco, Brazil 10 - 13 Sep 2017 London South Bank University.
On the use of skin texture features for gender recognition: An experimental evaluation
Bianconi, F, Smeraldi, F, Abdollahyan, M and Xiao, P (2017). On the use of skin texture features for gender recognition: An experimental evaluation. 6th International Conference on Image Processing Theory Tools and Applications (IPTA), 2016. Oulu, Finland 12 - 15 Dec 2016 IEEE. doi:10.1109/IPTA.2016.7821018
In vivo skin capacitive imaging analysis by using grey level co-occurrence matrix (GLCM).
Ou, X, Pan, W and Xiao, P (2013). In vivo skin capacitive imaging analysis by using grey level co-occurrence matrix (GLCM). International Journal of Pharmaceutics. 460 (1-2), pp. 28 - 32.
The occlusion effects in capacitive contact imaging for in vivo skin damage assessments.
Pan, W, Zhang, X, Lane, ME and Xiao, P (2015). The occlusion effects in capacitive contact imaging for in vivo skin damage assessments. International Journal of Cosmetic Science. 37 (4), pp. 395 - 400.
Multi-Location Clinical Trials: Do TEWL Readings Change With Altitude?
Kramer, G, Xiao, P, Crowther, J and Imhof, RE (2015). Multi-Location Clinical Trials: Do TEWL Readings Change With Altitude? Scientific meeting & technology showcase. New York 10 - 11 Dec 2015 London South Bank University.
Personal identification based on skin texture features from the forearm and multi-modal imaging
Bianconi, F, Chirikhina, E, Smeraldi, F, Bontozoglou, C and Xiao, P (2016). Personal identification based on skin texture features from the forearm and multi-modal imaging. Skin Research and Technology. 23 (3), pp. 392-398.
Membrane solvent penetration measurements using contact imaging
Xiao, P, Abdalghafor, H and Lane, ME (2013). Membrane solvent penetration measurements using contact imaging. in: Chilcott, R and Brain, K (ed.) Advances in Dermatological Sciences London, UK Royal Society of Chemistry.
BIG DATA ANALYTICS IN THE PUBLIC SECTOR: A CASE STUDY OF NEET ANALYSIS FOR THE LONDON BOROUGHS
Chen, D, Asaolu, B and Qin, C (2016). BIG DATA ANALYTICS IN THE PUBLIC SECTOR: A CASE STUDY OF NEET ANALYSIS FOR THE LONDON BOROUGHS. International Conference on Big Data Analytics, Data Mining and Computational Intelligence. Funchal, Madeira, Portugal 02 - 04 Jul 2016 London South Bank University.
On Distributed Deep Network for Processing Large-Scale Sets of Complex Data
Chen, D (2016). On Distributed Deep Network for Processing Large-Scale Sets of Complex Data. 8th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC). Hangzhou, China 27 - 28 Aug 2016 London South Bank University.
Photothermal Radiometry for Skin Research
Xiao, P (2016). Photothermal Radiometry for Skin Research. Cosmetics. 3 (1), p. 10.
Capacitive Contact Imaging For Skin Characterization
Xiao, P, Zhang, X and Bontozoglou, C (2016). Capacitive Contact Imaging For Skin Characterization. Perspectives in Percutaneous Penetration 2016. La Grande Motte, France 29 Mar - 01 Apr 2016 London South Bank University.
Skin Image Retrieval Using Gabor Wavelet Texture Feature
Ou, X, Pan, W, Zhang, X and Xiao, P (2016). Skin Image Retrieval Using Gabor Wavelet Texture Feature. International Journal of Cosmetic Science. 38 (6), pp. 607-614.
Hair Water Content and Water Holding Capacity Measurements
Xiao, P, Bontozoglou, C, Ciortea, LI and Imhof, RE (2016). Hair Water Content and Water Holding Capacity Measurements. 7th International Bi-Annual Conference on Applied Hair Science. Red Bank, NJ, USA 08 - 09 Jun 2016 London South Bank University.
Capacitive Imaging For Skin Characterization and Solvent Penetration
Xiao, P, Zhang, X and Bontozoglou, C (2016). Capacitive Imaging For Skin Characterization and Solvent Penetration. Skin Forum 2016 Annual Meeting. London London South Bank University.