A novel Supervised t-SNE based approach of viseme classification for automated lip reading
Fenghour, S., Chen, D., Hajderanj, L., Weheliye, I. and Xiao, P. (2021). A novel Supervised t-SNE based approach of viseme classification for automated lip reading. IEEE International Conference on Electrical, Computer and Energy Technologies. 09 - 10 Dec 2021 IEEEExplore.
|Authors||Fenghour, S., Chen, D., Hajderanj, L., Weheliye, I. and Xiao, P.|
Visemes are the most fundamental unit of visual speech and so viseme classification plays an important role in automated lip reading. Training a CNN-based classifier is usually time-consuming along with uncertain network topology. In this paper, a novel approach to viseme classification is proposed. The main idea of the approach is to first map the original high dimensional imagery data into a two dimensional space by using Supervised t-Distributed Stochastic Neighbour Embedding, and then conduct classification in the low dimensional space. The effectiveness of the proposed approach has been demonstrated by classifying visemes of three different frame widths, with an average accuracy, 98.5%, 94.0%, and 82.1%, respectively. Correspondingly, in comparison, CNN-based classifiers have archived an average accuracy of 66.2%, 75.2%, and 84.4%, respectively. In addition, the new approach has taken much less CPU time for training.
|Keywords||lip reading, speech recognition, neural networks, visemes, convolutional neural networks, supervised t-SNE, k-Nearest Neighbours|
|Web address (URL)||https://www.icecet.com|
|Accepted author manuscript|
File Access Level
|10 Dec 2021|
|Publication process dates|
|Accepted||25 Oct 2021|
|Deposited||22 Nov 2021|
Accepted author manuscript
0views this month
0downloads this month