The Sound of the Smell (and Taste) of My Shoes Too: Mapping the Senses Using Emotion as a Medium

Conference paper


Cunningham, S and Weinel, J (2016). The Sound of the Smell (and Taste) of My Shoes Too: Mapping the Senses Using Emotion as a Medium. Audio Mostly 2016 (AM 2016). Norrköping, Sweden. 04 - 06 Oct 2016 Association for Computing Machinery. pp. 28-33 doi:10.1145/2986416.2986456
AuthorsCunningham, S and Weinel, J
TypeConference paper
Abstract

This work discusses basic human senses: sight; sound; touch; taste; and smell; and the way in which it may be possible to compensate for lack of one, or more, of these by explicitly representing stimuli using the remaining senses. There may be many situations or scenarios where not all five of these base senses are being stimulated, either because of an optional restriction or deficit or because of a physical or sensory impairment such as loss of sight or touch sensation. Related to this there are other scenarios where sensory matching problems may occur. For example: a user immersed in a virtual environment may have a sense of smell from the real world that is unconnected to the virtual world. In particular, this paper is concerned with how sound can be used to compensate
for the lack of other sensory stimulation and vice-versa. As a link
is well established already between the visual, touch, and auditory systems, more attention is given to taste and smell, and their relationship with sound. This work presents theoretical concepts, largely oriented around mapping other sensory qualities to sound, based upon existing work in the literature and emerging technologies, to discuss where particular gaps currently exist, how emotion could be a medium to cross-modal representations, and how these might be addressed in future research. It is postulated that descriptive qualities, such as timbre or emotion, are currently the most viable routes for further study and that this may be later
integrated with the wider body of research into sensory
augmentation.

Year2016
JournalAM ’16 Proceedings of the Audio Mostly 2016
PublisherAssociation for Computing Machinery
Digital Object Identifier (DOI)doi:10.1145/2986416.2986456
Accepted author manuscript
License
CC BY 4.0
Publication dates
Print04 Oct 2016
Publication process dates
Deposited02 Mar 2019
Accepted04 Oct 2016
ISBN978-1-4503-4822-5
Page range28-33
Permalink -

https://openresearch.lsbu.ac.uk/item/871zy

  • 2
    total views
  • 22
    total downloads
  • 1
    views this month
  • 2
    downloads this month

Related outputs

Altered states of consciousness as an adaptive principle for composing electroacoustic music
Weinel, J (2012). Altered states of consciousness as an adaptive principle for composing electroacoustic music. PhD Thesis Keele University
Space Temple
Weinel, J (2017). Space Temple. Birmingham University
EVA London (Electronic Visualisation and the Arts): Proceedings of EVA London 2019
Weinel, J, Bowen, JP, Diprose, G and Lambert, N (2019). EVA London (Electronic Visualisation and the Arts): Proceedings of EVA London 2019. British Computer Society.
Sound Through The Rabbit Hole: Sound Design Based On Reports of Auditory Hallucination
Weinel, J, Cunningham, S and Griffiths, D (2014). Sound Through The Rabbit Hole: Sound Design Based On Reports of Auditory Hallucination. 9th Audio Mostly: A Conference on Interaction With Sound. Aalborg, Denmark 01 - 03 Oct 2014 Association for Computing Machinery. doi:10.1145/2636879.2636883
ACERemix: A Tool for Glitch Music Production and Performance
Cunningham, S, Weinel, J and Griffiths, D (2014). ACERemix: A Tool for Glitch Music Production and Performance. 9th Audio Mostly: A Conference on Interaction With Sound. Aalborg, Denmark 01 - 03 Oct 2014 Association for Computing Machinery. doi:10.1145/2636879.2636888
Quake Delirium EEG: A Pilot Study Regarding Biofeedback-Driven Visual Effects in a Computer Game
Weinel, J, Cunningham, S, Roberts, N, Griffiths, D and Roberts, S (2015). Quake Delirium EEG: A Pilot Study Regarding Biofeedback-Driven Visual Effects in a Computer Game. 2015 Internet Technologies and Applications (ITA). Wrexham, UK 08 - 11 Sep 2015 IEEE. doi:10.1109/ITechA.2015.7317420
Second Screen Comes to the Silver Screen: A Consumer Study Regarding Mobile Technologies in the Cinema
Weinel, J and Cunningham, S (2015). Second Screen Comes to the Silver Screen: A Consumer Study Regarding Mobile Technologies in the Cinema. 2015 Internet Technologies and Applications (ITA). Wrexham, UK 08 - 11 Sep 2015 IEEE. doi:10.1109/ITechA.2015.7317381
Second Screen Comes to the Silver Screen: A Technology Feasibility Study Regarding Mobile Technologies in the Cinema
Cunningham, S and Weinel, J (2015). Second Screen Comes to the Silver Screen: A Technology Feasibility Study Regarding Mobile Technologies in the Cinema. 2015 Internet Technologies and Applications (ITA). Wrexham, UK 08 - 11 Sep 2015 IEEE. doi:10.1109/ITechA.2015.7317400
High-Level Analysis of Audio Features for Identifying Emotional Valence in Human Singing
Cunningham, S, Weinel, J and Picking, R (2018). High-Level Analysis of Audio Features for Identifying Emotional Valence in Human Singing. Audio Mostly 2018: A conference on interaction with sound. Wrexham Glyndŵr University, North Wales, UK 12 - 14 Sep 2018 doi:10.1145/3243274.3243313
In-Game Intoxication: Demonstrating the Evaluation of the Audio Experience of Games with a Focus on Altered States of Consciousness
Cunningham, S, Weinel, J and Picking, R (2016). In-Game Intoxication: Demonstrating the Evaluation of the Audio Experience of Games with a Focus on Altered States of Consciousness. in: Garcia-Ruiz, M (ed.) Games User Research: A Case Study Approach New York CRC Press. pp. 97-118
EVA London 2018
Bowen, J, Giannini, T, Polmeer, G, Gannis, C, Gardiner, J, Kearney, K, Wands, B and Weinel, J Bowen, JP, Weinel, J, Diprose, G and Lambert, N (ed.) (2018). EVA London 2018. British Computer Society.
Deep Subjectivity and Empathy in Virtual Reality: A Case Study on the Autism TMI Virtual Reality Experience
Weinel, J, Cunningham, S and Pickles, J (2019). Deep Subjectivity and Empathy in Virtual Reality: A Case Study on the Autism TMI Virtual Reality Experience. in: Filimowicz, M and Tzankova, V (ed.) New Directions in Third Wave HCI Springer.
Designing Game Audio Based on Avatar-Centred Subjectivity
Weinel, J and Cunningham, S (2019). Designing Game Audio Based on Avatar-Centred Subjectivity. in: Filimowicz, M (ed.) Foundations in Sound Design for Interactive Media Routledge.
Augmented Unreality: Synesthetic Artworks & Audio-Visual Hallucinations
Weinel, J (2019). Augmented Unreality: Synesthetic Artworks & Audio-Visual Hallucinations. in: Grimshaw, M, Walther-Hansen, M and Knakkergaard, M (ed.) The Oxford Handbook of Sound & Imagination Oxford Oxford University Press.
Tiny Jungle: Psychedelic Techniques in Audio-Visual Composition
Weinel, J (2011). Tiny Jungle: Psychedelic Techniques in Audio-Visual Composition. 2011 International Computer Music Conference (ICMC). Huddersfield, UK 31 Jul - 05 Aug 2011
Shamanic diffusions: a technoshamanic philosophy of electroacoustic music
Weinel, J (2014). Shamanic diffusions: a technoshamanic philosophy of electroacoustic music. Sonic Ideas/Ideas Sonicas. 6 (12).
Holophonor: On the Future Technology of Visual Music
Weinel, J, Cunningham, S, Picking, R and Williams, L (2015). Holophonor: On the Future Technology of Visual Music. in: Curran, K (ed.) Recent Advances in Ambient Intelligence and Context-Aware Computing IGI Global. pp. 248-261
Representing Altered States of Consciousness in Computer Arts
Weinel, J (2015). Representing Altered States of Consciousness in Computer Arts. Ng, Kia, Bowen, Jonathan P and Lambert, Nick (ed.) Electronic Visualisation and the Arts (EVA 2015). London, UK 07 - 09 Jul 2015 British Computer Society.
An Interactive Music Playlist Generator that Responds to User Emotion and Context
Griffiths, D, Cunningham, S and Weinel, J (2016). An Interactive Music Playlist Generator that Responds to User Emotion and Context. Bowen, Jonathan P, Diprose, Graham and Lambert, Nick (ed.) Electronic Visualisation and the Arts (EVA 2016). London, UK 12 - 14 Jul 2016 British Computer Society. doi:10.14236/ewic/EVA2016.53
Simulating Auditory Hallucinations in a Video Game: Three Prototype Mechanisms
Weinel, J and Cunningham, S (2017). Simulating Auditory Hallucinations in a Video Game: Three Prototype Mechanisms. Audio Mostly 2017. London 23 - 26 Aug 2017 Association for Computing Machinery. doi:10.1145/3123514.3123532
Technoshamanic Visions from the Underworld II
Weinel, J (2017). Technoshamanic Visions from the Underworld II. in: Cunningham, S, Picking, R, Houlden, N, Oram, D, Grout, V and Mayers, J (ed.) 2017 Internet Technologies and Applications (ITA) Proceedings of the Seventh International Conference United States IEEE. pp. 345-346
Inner Sound: Altered States of Consciousness in Electronic Music and Audio-Visual Media
Weinel, J (2018). Inner Sound: Altered States of Consciousness in Electronic Music and Audio-Visual Media. Oxford Oxford University Press.
States of Being: Art and identity in digital space and time
Bowen, J, Giannini, T, Polmeer, G, Gannis, C, Gardiner, J, Kearney, J, Wands, B and Weinel, J (2018). States of Being: Art and identity in digital space and time. EVA London Symposium. London, UK British Computer Society. pp. 1–8-1–8
Bass Drum, Saxophone and Laptop: Real-time Psychedelic Performance Software
Weinel, J (2010). Bass Drum, Saxophone and Laptop: Real-time Psychedelic Performance Software. eContact. 12 (4).
Mutations (megamix): exploring notions of the ‘DJ set’, ‘mashup’ and ‘remix’ through live piano-based performance
Ratcliffe, R, Weinel, J and Kanga, Z (2011). Mutations (megamix): exploring notions of the ‘DJ set’, ‘mashup’ and ‘remix’ through live piano-based performance. eContact. 13 (2).
Digitized Direct Animation: Creating Materials for Electroacoustic Visual Music Using 8mm Film
Weinel, J and Cunningham, S (2014). Digitized Direct Animation: Creating Materials for Electroacoustic Visual Music Using 8mm Film. eContact. 15 (4).
EEG as a Controller for Psychedelic Visual Music in an Immersive Dome Environment
Weinel, J, Cunningham, S, Roberts, N, Roberts, S and Griffiths, D (2014). EEG as a Controller for Psychedelic Visual Music in an Immersive Dome Environment. Ng, Kia, Bowen, Jonathan and McDaid, Sarah (ed.) EVA London 2014: Electronic Visualisation & the Arts. London, UK 08 - 10 Jul 2014 British Computer Society.
Easter Eggs: Hidden Tracks and Messages in Musical Mediums
Weinel, J, Griffiths, D and Cunningham, S (2014). Easter Eggs: Hidden Tracks and Messages in Musical Mediums. Joint Conference ICMC-SMC 2014. Athens, Greece 14 - 20 Sep 2014 International Computer Music Association.
Affective Audio
Weinel, J, Cunningham, S, Griffiths, D, Roberts, S and Picking, R (2014). Affective Audio. Leonardo. 24 (24), pp. 17-20.
Optical Research: A Curated Visual Music Collection
Weinel, J (2015). Optical Research: A Curated Visual Music Collection. Ng, Kia, Bowen, Jonathan P and Lambert, Nick (ed.) (EVA 2015) Electronic Visualisation and the Arts. London, UK 07 - 09 Jul 2015 British Computer Society.
Entoptic Phenomena in Audio: Categories of Psychedelic Electroacoustic Composition
Weinel, J (2016). Entoptic Phenomena in Audio: Categories of Psychedelic Electroacoustic Composition. Contemporary Music Review. 35 (2), pp. 202-223.