Saliency tubes: visual explanations for spatio-temporal convolutions

Conference paper


Stergiou, A, Kapidis, G, Kalliatakis, G, Chrysoulas, C, Veltkamp, R and Poppe, R (2019). Saliency tubes: visual explanations for spatio-temporal convolutions. 26th IEEE International Conference on Image Processing (ICIP). Taipei, Taiwan 22 - 25 Sep 2019
AuthorsStergiou, A, Kapidis, G, Kalliatakis, G, Chrysoulas, C, Veltkamp, R and Poppe, R
TypeConference paper
Abstract

Deep learning approaches have been established as the
main methodology for video classification and recognition.
Recently, 3-dimensional convolutions have been used to
achieve state-of-the-art performance in many challenging
video datasets. Because of the high level of complexity of
these methods, as the convolution operations are also extended
to an additional dimension in order to extract features
from it as well, providing a visualization for the signals that
the network interpret as informative, is a challenging task.
An effective notion of understanding the network’s innerworkings
would be to isolate the spatio-temporal regions on
the video that the network finds most informative. We propose
a method called Saliency Tubes which demonstrate the
foremost points and regions in both frame level and over time
that are found to be the main focus points of the network. We
demonstrate our findings on widely used datasets for thirdperson
and egocentric action classification and enhance the
set of methods and visualizations that improve 3D Convolutional
Neural Networks (CNNs) intelligibility. Our code 1
and a demo video 2 are also available.

KeywordsVisual explanations; explainable convolutions; spatio-temporal feature representation
Year2019
Accepted author manuscript
License
CC BY 4.0
Publication process dates
Deposited20 Jun 2019
Accepted01 May 2019
Permalink -

https://openresearch.lsbu.ac.uk/item/866vv

Accepted author manuscript

  • 1
    total views
  • 0
    total downloads
  • 1
    views this month
  • 0
    downloads this month

Related outputs

Authentication Layer for IEC 61113-3 Applications
Chrysoulas, C, Aydin, H and Mario, S (2019). Authentication Layer for IEC 61113-3 Applications. 10th IFIP International Conference on New Technologies, Mobility & Security. Gran Canaria, Spain 24 - 26 Jun 2019
ProMe: A Mentoring Platform for Older Adults Using Machine Learning Techniques for Supporting the “Live and Learn” Concept
Chrysoulas, C, Kostopoulos, G, Neureiter, K, Papatoiu, D and Tscheligi, M (2018). ProMe: A Mentoring Platform for Older Adults Using Machine Learning Techniques for Supporting the “Live and Learn” Concept. Mobile Information Systems. 2018.
Traffic Sign Recognition based on Synthesised Training Data
Stergiou, A, Kalliatakis, G and Chrysoulas, C (2018). Traffic Sign Recognition based on Synthesised Training Data. Big Data and Cognitive Computing. 2 (3), p. 19.
Towards an adaptive SOA-based QoS & Demand-Response Provisioning Architecture for the Smart Grid
Chrysoulas, C and Fasli, M (2017). Towards an adaptive SOA-based QoS & Demand-Response Provisioning Architecture for the Smart Grid. Journal of Communications Software and Systems. 13 (2), pp. 77-86.
Hadoop and what it is good for
Chrysoulas, C, Kalliatakis, G and Stamatiadis, G (2017). Hadoop and what it is good for. in: Inge, R and Leif, J (ed.) Machine Learning: Advances in Research and Applications Nova Science Publishers, Inc.. pp. 177-194
A service oriented QoS architecture targeting the smart grid world & machine learning aspects
Chrysoulas, C and Fasli, M (2016). A service oriented QoS architecture targeting the smart grid world & machine learning aspects. 2016 International Multidisciplinary Conference On Computer And Energy Science (SpliTech). Split, Croatia 13 - 15 Jul 2016 London South Bank University. doi:10.1109/SpliTech.2016.7555923
Arrowhead compliant virtual market of energy
Ferreira, LL, Siksnys, L, Pedersen, P, Stluka, P, Chrysoulas, C, Le Guilly, T, Albano, M, Skou, A, Teixeira, C and Pedersen, T (2014). Arrowhead compliant virtual market of energy. Emerging Technology and Factory Automation. Barcelona, Spain 16 - 19 Sep 2014 London South Bank University. doi:10.1109/ETFA.2014.7005193
The arrowhead approach for SOA application development and documentation
Blomstedt, F, Ferreira, LL, Klisics, M, Chrysoulas, C, De Soria, IM, Morin, B, Zabasta, A, Eliasson, J, Johansson, M and Varga, P (2014). The arrowhead approach for SOA application development and documentation. IECON 2014 - 40th Annual Conference of the IEEE Industrial Electronics Society. Dallas, Texas 29 Oct - 01 Nov 2014 London South Bank University. pp. 2631-2637 doi:10.1109/IECON.2014.7048877
Exploiting voting strategies in partially replicated IEC 61499 applications
De Sousa, M, Chrysoulas, C and Homay, A (2015). Exploiting voting strategies in partially replicated IEC 61499 applications. 2015 IEEE World Conference on Factory Communication Systems. Palma de Mallorca, Spain 27 - 29 May 2015 London South Bank University. doi:10.1109/WFCS.2015.7160589
Multiply and conquer: A replication framework for building fault tolerant industrial applications
De Sousa, M, Chrysoulas, C and Homay, A (2015). Multiply and conquer: A replication framework for building fault tolerant industrial applications. 2015 IEEE 13th International Conference on Industrial Informatics. Cambridge, UK 22 - 24 Jul 2015 London South Bank University. pp. 1342-1347 doi:10.1109/INDIN.2015.7281930
Tolerating partial failures on IEC 61499 applications
De Sousa, M, Chrysoulas, C and Homay, A (2015). Tolerating partial failures on IEC 61499 applications. ACM SIGBED Review. 12 (3), pp. 32-35.
Towards a Security Enabled and SOA-based QoS (for the Smart Grid) Architecture.
Chrysoulas, C and Pitropakis, N (2018). Towards a Security Enabled and SOA-based QoS (for the Smart Grid) Architecture. EAI Endorsed Transactions on Industrial Networks and Intelligent Systems. 4, pp. e4-e4.
QoS provisioning in a service-oriented architecture targeting the smart grid world
Chrysoulas, C (2018). QoS provisioning in a service-oriented architecture targeting the smart grid world. International Journal of Smart Grid and Green Communications. 1 (3), pp. 216-226.
Teaching Industrial Automation Concepts with the use of Virtual/Augmented Reality – The IEC 61499 Case
Chrysoulas, C, Homay, A and Lemac, M (2018). Teaching Industrial Automation Concepts with the use of Virtual/Augmented Reality – The IEC 61499 Case. ITHET 2018, 17th International Conference on Information Technology Based Higher Education and Training. Olhao, Portugal 26 - 28 Apr 2018 London South Bank University.
Building an adaptive E-learning system
Chrysoulas, C and Fasli, M (2017). Building an adaptive E-learning system. 9th International Conference on Computer Supported Education. Porto, Portugal 21 - 23 Apr 2017 London South Bank University.