Saliency tubes: visual explanations for spatio-temporal convolutions

Conference paper


Stergiou, A, Kapidis, G, Kalliatakis, G, Chrysoulas, C, Veltkamp, R and Poppe, R (2019). Saliency tubes: visual explanations for spatio-temporal convolutions. 26th IEEE International Conference on Image Processing (ICIP). Taipei, Taiwan 22 - 25 Sep 2019
AuthorsStergiou, A, Kapidis, G, Kalliatakis, G, Chrysoulas, C, Veltkamp, R and Poppe, R
TypeConference paper
Abstract

Deep learning approaches have been established as the
main methodology for video classification and recognition.
Recently, 3-dimensional convolutions have been used to
achieve state-of-the-art performance in many challenging
video datasets. Because of the high level of complexity of
these methods, as the convolution operations are also extended
to an additional dimension in order to extract features
from it as well, providing a visualization for the signals that
the network interpret as informative, is a challenging task.
An effective notion of understanding the network’s innerworkings
would be to isolate the spatio-temporal regions on
the video that the network finds most informative. We propose
a method called Saliency Tubes which demonstrate the
foremost points and regions in both frame level and over time
that are found to be the main focus points of the network. We
demonstrate our findings on widely used datasets for thirdperson
and egocentric action classification and enhance the
set of methods and visualizations that improve 3D Convolutional
Neural Networks (CNNs) intelligibility. Our code 1
and a demo video 2 are also available.

KeywordsVisual explanations; explainable convolutions; spatio-temporal feature representation
Year2019
Accepted author manuscript
License
File Access Level
Open
Publication dates
Print29 Sep 2019
Publication process dates
Deposited20 Jun 2019
Accepted01 May 2019
Permalink -

https://openresearch.lsbu.ac.uk/item/866vv

Download files


Accepted author manuscript
  • 140
    total views
  • 245
    total downloads
  • 1
    views this month
  • 0
    downloads this month

Export as

Related outputs

Information Fusion for 5G IoT: An Improved 3D Localisation Approach Using K-DNN and Multi-Layered Hybrid Radiomap
El Boudani, B., Dagiuklas, A., Kanaris, L., Iqbal, M. and Chrysoulas, C. (2023). Information Fusion for 5G IoT: An Improved 3D Localisation Approach Using K-DNN and Multi-Layered Hybrid Radiomap. Electronics. 12 (19), p. 4150. https://doi.org/10.3390/electronics12194150
Positioning as Service for 5G IoT Networks
El Boudani, B., Kanaris, L., Kokkinis, A., Chrysoulas, C., Dagiuklas, A. and Stavrou, S. (2021). Positioning as Service for 5G IoT Networks. 2021 Telecoms Conference (ConfTELE). Leiria 10 - 12 Feb 2021 Institute of Electrical and Electronics Engineers (IEEE).
Measuring consumer behavioural intention to accept technology: Towards autonomous vehicles technology acceptance model (AVTAM)
Ubakanma, G., Seuwou, P., Chrysoulas, C. and Banissi, E. (2020). Measuring consumer behavioural intention to accept technology: Towards autonomous vehicles technology acceptance model (AVTAM). WorldCIST: World Conference on Information Systems and Technologies. Budva, Montenegro 07 - 10 Apr 2020 Springer. https://doi.org/10.1007/978-3-030-45688-7
Implementing Deep Learning Techniques in 5G IoT Networks for 3D Indoor Positioning: DELTA (DeEp Learning-Based Co-operaTive Architecture)
El Boudani, B., Kanaris, L., Kokkinis, A., Kyriacou, M., Chrysoulas, C., Stavrou, S. and Dagiuklas, A. (2020). Implementing Deep Learning Techniques in 5G IoT Networks for 3D Indoor Positioning: DELTA (DeEp Learning-Based Co-operaTive Architecture). Sensors. 20 (19), p. e5495. https://doi.org/10.3390/s20195495
Conceptualising Green Awareness as Moderator in Technology Acceptance Model for Green IS/IT
Ashiq, S., Banissi, E. and Chrysoulas, C. (2019). Conceptualising Green Awareness as Moderator in Technology Acceptance Model for Green IS/IT. 2019 International Conference on Innovative Computing (ICIC). 01 - 02 Nov 2019 Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/ICIC48496.2019.8966710
Authentication Layer for IEC 61113-3 Applications
Chrysoulas, C, Aydin, H and Mario, S (2019). Authentication Layer for IEC 61113-3 Applications. 10th IFIP International Conference on New Technologies, Mobility & Security. Gran Canaria, Spain 24 - 26 Jun 2019 https://doi.org/10.1109/NTMS.2019.8763839
Hadoop and what it is good for
Chrysoulas, C, Kalliatakis, G and Stamatiadis, G (2017). Hadoop and what it is good for. in: Inge, R and Leif, J (ed.) Machine Learning: Advances in Research and Applications Nova Science Publishers, Inc.. pp. 177-194
Towards a Security Enabled and SOA-based QoS (for the Smart Grid) Architecture.
Chrysoulas, C and Pitropakis, N (2018). Towards a Security Enabled and SOA-based QoS (for the Smart Grid) Architecture. EAI Endorsed Transactions on Industrial Networks and Intelligent Systems. 4, pp. e4-e4. https://doi.org/10.4108/eai.10-1-2018.153551
QoS provisioning in a service-oriented architecture targeting the smart grid world
Chrysoulas, C (2018). QoS provisioning in a service-oriented architecture targeting the smart grid world. International Journal of Smart Grid and Green Communications. 1 (3), pp. 216-226. https://doi.org/10.1504/IJSGGC.2018.091350
Teaching Industrial Automation Concepts with the use of Virtual/Augmented Reality – The IEC 61499 Case
Chrysoulas, C, Homay, A and Lemac, M (2018). Teaching Industrial Automation Concepts with the use of Virtual/Augmented Reality – The IEC 61499 Case. ITHET 2018, 17th International Conference on Information Technology Based Higher Education and Training. Olhao, Portugal 26 - 28 Apr 2018
Traffic Sign Recognition based on Synthesised Training Data
Stergiou, A, Kalliatakis, G and Chrysoulas, C (2018). Traffic Sign Recognition based on Synthesised Training Data. Big Data and Cognitive Computing. 2 (3), p. 19. https://doi.org/10.3390/bdcc2030019
ProMe: A Mentoring Platform for Older Adults Using Machine Learning Techniques for Supporting the “Live and Learn” Concept
Chrysoulas, C, Kostopoulos, G, Neureiter, K, Papatoiu, D and Tscheligi, M (2018). ProMe: A Mentoring Platform for Older Adults Using Machine Learning Techniques for Supporting the “Live and Learn” Concept. Mobile Information Systems. 2018. https://doi.org/10.1155/2018/9723268
Building an adaptive E-learning system
Chrysoulas, C and Fasli, M (2017). Building an adaptive E-learning system. 9th International Conference on Computer Supported Education. Porto, Portugal 21 - 23 Apr 2017
Towards an adaptive SOA-based QoS & Demand-Response Provisioning Architecture for the Smart Grid
Chrysoulas, C and Fasli, M (2017). Towards an adaptive SOA-based QoS & Demand-Response Provisioning Architecture for the Smart Grid. Journal of Communications Software and Systems. 13 (2), pp. 77-86. https://doi.org/10.24138/jcomss.v13i2.375
A service oriented QoS architecture targeting the smart grid world & machine learning aspects
Chrysoulas, C and Fasli, M (2016). A service oriented QoS architecture targeting the smart grid world & machine learning aspects. 2016 International Multidisciplinary Conference On Computer And Energy Science (SpliTech). Split, Croatia 13 - 15 Jul 2016 Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/SpliTech.2016.7555923
Exploiting voting strategies in partially replicated IEC 61499 applications
De Sousa, M, Chrysoulas, C and Homay, A (2015). Exploiting voting strategies in partially replicated IEC 61499 applications. 2015 IEEE World Conference on Factory Communication Systems. Palma de Mallorca, Spain 27 - 29 May 2015 Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/WFCS.2015.7160589
Multiply and conquer: A replication framework for building fault tolerant industrial applications
De Sousa, M, Chrysoulas, C and Homay, A (2015). Multiply and conquer: A replication framework for building fault tolerant industrial applications. 2015 IEEE 13th International Conference on Industrial Informatics. Cambridge, UK 22 - 24 Jul 2015 Institute of Electrical and Electronics Engineers (IEEE). pp. 1342-1347 https://doi.org/10.1109/INDIN.2015.7281930
Tolerating partial failures on IEC 61499 applications
De Sousa, M, Chrysoulas, C and Homay, A (2015). Tolerating partial failures on IEC 61499 applications. ACM SIGBED Review. 12 (3), pp. 32-35. https://doi.org/10.1145/2815482.2815488
Arrowhead compliant virtual market of energy
Ferreira, LL, Siksnys, L, Pedersen, P, Stluka, P, Chrysoulas, C, Le Guilly, T, Albano, M, Skou, A, Teixeira, C and Pedersen, T (2014). Arrowhead compliant virtual market of energy. Emerging Technology and Factory Automation. Barcelona, Spain 16 - 19 Sep 2014 Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/ETFA.2014.7005193
The arrowhead approach for SOA application development and documentation
Blomstedt, F, Ferreira, LL, Klisics, M, Chrysoulas, C, De Soria, IM, Morin, B, Zabasta, A, Eliasson, J, Johansson, M and Varga, P (2014). The arrowhead approach for SOA application development and documentation. IECON 2014 - 40th Annual Conference of the IEEE Industrial Electronics Society. Dallas, Texas 29 Oct - 01 Nov 2014 Institute of Electrical and Electronics Engineers (IEEE). pp. 2631-2637 https://doi.org/10.1109/IECON.2014.7048877