Digital Arts and New Media: MFA: Collaboration, Innovation, Social Impact

You are here

The Role of Sound in Integrating Art, Science, and the Environment (winter - fall 2016)

David Dunn


UCSC is well established as having one of the most outstandingly beautiful university campuses in the world. Besides its natural splendor, it is home to hundreds of diverse animal and plant species, including invertebrate species that occur nowhere else on earth. Furthermore its direct proximity to the redwood forests of the Santa Cruz Mountains and the spectacular marine ecology of the Monterey Bay, make it one of the most unique and appropriate locales for researching the role that sound communication and phenomena have in integrating complex natural and human ecologies. It has a near infinite potential for both research opportunities and creative projects involving student participation in creating new approaches for environmental sound research, creative performances and art/science collaboration.

This research agenda will focus upon creating diverse opportunities to investigate new methods for increasing the necessary monitoring of our environment through sound,  facilitate an increase in our collective environmental sensitivity, discovery of unknown natural and human-made sound phenomena, provide novel inexpensive audio tools for both artists and scientists, and contribute towards practical environmental problem solving. 

Having our aural sense expanded through technology allows our ears to be more on a par with other forms of life that surround us. The advent of digital audio has strengthened this potential as has never been previously possible. The focusing of such technology towards this expansion of consciousness, therefore, has an additional benefit: it gives us access to listening beyond the boundaries of our usual human perception. It applies current technological breakthroughs in digital sound recording and manipulation towards a nonhuman centered and environmentally relevant art practice. While most art making harvests the imaginative potential of the overlap between biological instinct and cultural values—both of which have co-evolved as intertwining genetic and epigenetic streams—my argument here has more to do with the historical necessity of where we currently stand in relationship to an earth in crises. Can art participate in the discovery of solutions that can accelerate or extend those of science? This requires a merger of art and science that places the human back into a measured position within the biotic world and encourages both to contribute to a collective environmental activism. This is an art requiring a dance between metaphor and mechanism. The art world needs to ground imagination in a deeper understanding of the natural world, and science needs to reach out beyond the desiccation of professional constraints in order to transform the epistemic errors of the world at large. Perhaps Gregory Bateson said it best: “Rigor alone is paralytic death, but imagination alone is insanity.”

Most attempted collaboration between science and art has occurred at an abstract level. Through this research agenda the hope is to provide an exemplar of how artists and scientists can collaborate towards real world problem solving. Traditional approaches have relied upon a familiar interpretive function where art provides a richness of metaphor for the communication of science to the broader public while science provides new tools and technical knowledge for the arts. One of the polemics that underlies this project is an intentionally subversive attitude towards these familiar relationships. Can artistic insight prove itself to be an effective participant in the framing of a scientific hypothesis? Can artistic creativity originate technical solutions that are useful to artists, scientists, and a broader public? What is the potential for art and science collaboration to not only interpret and communicate empirical scientific data as cultural product but also delineate a research agenda that is worthy of serious scientific investigation? Is it possible for art/science collaboration to fulfill the disciplinary criteria of both in a non-trivial manner?


Excerpt from Winter 2016 Syllabus:

DANM 250A - 01 Collaborative Research Project Group: Mechatronics
Full Spectrum Acoustic Portrait of an Ecosystem
Instructor: David Dunn / Research Assistant: Andrea Steves
Room: DARC 217
Syllabus: Winter 2016


The project involves the creation of separate and simultaneous multi-channel recordings of aquatic and terrestrial aspects of ecosystems, inclusive of infrasonic, normal and ultrasonic acoustic, diverse vibratory substrates, and natural radio emissions; to analyze correlations between these ranges and their generative sources; and to present the resultant recorded materials in an experiential, and educational context through a high-resolution performance/ exhibition and published analytical summary.

Location to Explore:
The site to be explored and documented will be determined during the first term of the three quarters assigned for the research project. All participants in the group will be required to research and record a variety of potential sites on or near the UCSC campus and build many of the transducer systems to be used in this process. Besides defining the requisite final project site, this activity will provide ample opportunities for participants to gain essential knowledge about the necessary tools and techniques appropriate for field and location audio recording.

For over 40 years my work as an artist/composer has foregrounded both issues of environmental health and auditory perception. It has also progressively moved towards the simultaneous fulfillment of both artistic and scientific criteria. In other words, I have produced a series of works that not only provide aesthetic and artistic descriptions of the natural world but also strive to have substantial scientific value. I have adopted the phrase— originally invoked by Gregory Bateson—“the necessity of double description” to clarify this dual presence of both science and art as a technique to transcend the more familiar theoretical discourse that usually passes for art and science interaction.

Several years ago I spent time living on a houseboat deep into the Atchafalaya Basin of Louisiana. Nights were spent making forays into the swamp to record continuous night sounds without interruption. One of the most striking features of this sound world was the abrupt transition between distinct collectives of sound makers. One group would manifest for several hours and then suddenly fade to silence. After a brief period of quiescence, a new aggregate of sound makers replaced them. The dynamic quality of these dense soundscapes, their abrupt transitions, and fantastic spatial motion, impressed upon me a sense that—beyond the communicative agenda of individual living sound generators—there was some underlying emergent logic at work to drive them into a global patterning.

No scientific study of such global interactions and feedback between diverse sound makers within an ecological network has ever been attempted. This project is a step towards modeling the assumption of such an underlying emergent dynamical force and the potential to study pattern generating processes within natural systems whose behaviors necessarily cross the multiple sensory and perceptual boundaries of observers and instrumentation.

While there is currently a great deal of bioacoustics research focused upon the sensing and communicative behavior of living organisms—occurring in an acoustic frequency domain outside of direct human sensory perception— most of this activity has been limited to the study of species specific behaviors. For example, groundbreaking research has focused upon the infrasonic communication of elephants and whales. An ever-expanding frontier of knowledge is also being revealed about the large number of organisms that signal and monitor mechanical vibration in the ultrasonic range. These include many species of bats, cetaceans, rodents, birds, amphibians, fish, and invertebrates. While there has also been some discussion of interspecies communication in the ultrasonic range, such as the influence of bat echolocation on the evolution of moths and butterflies, there has been little significant work in the area of large-scale communicative interaction through and across the full acoustic spectrum of specific biohabitats.

Our purpose in pursuing this project is three-fold: 1) to ascertain whether or not such full spectrum acoustic portraits might constitute a viable technique for ecological monitoring through the revelation of emergent patterns of information that are not otherwise perceivable to researchers; 2) to contribute information about how different acoustic domains (i.e. aquatic versus terrestrial) structurally couple and exchange acoustic information; and 3) to create an exhibition/publication that not only represents the results of scientific research but is also an immersive aesthetic experience that guides us towards the realization of the limitations of our human senses and how that impacts our relationship to the natural environment. The exhibit/ publication will focus upon the question: what is happening in the living world beyond our senses and how might we best pursue a deeper understanding of their complexity?

Methodology and Tools:
The recording process will entail a number of existing field technologies such as digital audio recorders, digital video recorders, high-resolution microphones, ambisonic and other microphones, hydrophones, ultrasonic microphones, geophones, and laptop computers equipped with data acquisition cards capable of appropriate high frequency sampling rates and ultrasonic recording software. An appropriate synchronization strategy for post-analysis between the diverse recording formats will also need to be devised. The project will also need audio playback equipment that is appropriate for the output of full range reference signals to be used for later calibration between the diverse data sets.

Preliminary analytical methods will most likely involve techniques borrowed from dynamical systems theory that can describe various aspects of communicative networks having a large spatial footprint.

Final Exhibition/Performance:
Within an appropriate exhibition venue, much of these sounds will be familiar as conventional terrestrial field recordings, but much will also be comprised of sounds that occur outside of our normal perceptual capabilities: underwater audio, frequency-shifted infrasonic and ultrasonic audio, substrate vibrations occurring in soil, plants and detritus, and natural radio emissions from the atmosphere and other sources such as insects. This intensive sound exploration may be represented within an enveloping visual environment consisting of the projection of conventional video representations of the site, and computer graphic visualizations of sound elements showing various otherwise hidden correlations between phenomena. The overall effect will hopefully help to reveal the profound fabric of sonic phenomena that integrates natural systems but goes undetected by human observers.