“Do You Smile in Your Dreams?”
Timothy Gmeiner - Director & Composer // Mingyong Cheng - Visual Designer, Creative Contributor // Nathaniel Haering - Sound-Design & Spatialization
Artist Statement: Underneath the deepest of pain we often find an even deeper beauty that only exists because of the stakes it was developed under; our connections to each other and collective memory is what sends us on the search through this pain to experience that beauty.
Abstract: "Do You Smile In Your Dreams?" is a spatialized fixed media audiovisual installation that journeys through a mind with dementia to paint the walls with sound: music and words that poke through the fog and isolation to invite colorful memory and human connectivity.
Visual: Our visual design integrates open-source MRI videos and 3D brain-scanning models with a computational visual system, remapping the scientific data with artistic expression. A.I., viewed as a representation of our collective memory, gathers text-based inspiration and elements through an A.I. chatbot to co-create this piece’s visual elements.
Audio: This piece is founded on a fixed media composition formed to the narrative arc of emergent memory. Sonic cues are interwoven with musical elements. Specifically, a Cherokee rendition of “The Lord’s Prayer" is split into clips and used as source audio to trigger visual activity in the 3D brain-scan models. Real-time, spatialized, granular synthesis and exponential, filtered delays stochastically disperse manipulated fragments of voices throughout the listening/internal mind space to simulate neurons firing electrical impulses and synapses reaching out and attempting to form connections as memories begin to break through the enveloping mental fog; moving from obfuscation to clarity and back again as the sounds strike upon core memories and elicit the powerful myriad of emotions bound to them.
Audiovisual Reactivity: Visual properties of our brain-scanning models react to the amplitude of specific audio elements within the piece. An earlier iteration of this piece allows for real-time reactivity such that a trained model of an ongoing MRI scan sequence triggers the launch of an audio clip which in turn triggers a visual distortion of the MRI, indicating reception of an audio message. This real-time reactivity is an optional approach for presentation of our current iteration.
Presentation: This piece was presented in February 2023 in the Qualcomm Institute’s Calit2 Theater at UC-San Diego on a tile-displayed set of 32 55” screens with 7.1 multichannel audio.
Sources:
3D Brain-scan models: Edlow, Brian L. et al. (2019), Data from: 7 Tesla MRI of the ex vivo human brain at 100 micron resolution, Dryad, Dataset, https://doi.org/10.5061/dryad.119f80q
“The Lord’s Prayer” in Cherokee (author unknown): https://youtu.be/RnMluQy4EmA
Acknowledgements: UC-San Diego via the IDEAS Initiative and Crossing Boundaries program