Aesthetics in Spatial Audio and Generative Sound Design

Lately, I have been exploring spatial sound and generative sound design strategies. These research activities have taken place within the research context of University of Tampere, School of Information Sciences in the unit of TAUCHI. In this post I shed light on some of my research activities during the first half of this year.

Research Environment

The research group where I work – Speech-based & Pervassive Interaction Research Group, also known as SPI – has a very nice studio with 16 speaker spatial audio setup, offering a stimulating environment for research activities. In addition, in that studio we conduct research on ambient lighting and multi-screen projections. These modalities are studied in various research cases (Figure 1).

Audio Cable Jungle

Figure 1. SPI Research Group Studio: audio cable jungle

Our research group has been working on a Energy Land research case collaborating with Muova, which is a part of Aalto University, School of Art, Design and Architecture. Our role in this ongoing research case is to design gesture based interaction as well as to come up with immersive audio-visual representation techniques. Also, our part is to implement the needed technologies. My specific task has been to design the multichannel sound system and the audio content for our prototype. The design team – we together with the researchers from Muova – designed and developed the first version of the prototype installation.

Energy Land Installation Setup

During the spring 2012 an embodied game-like application was realized. The basic idea of the interactive application is to represent novel smart energy concepts through audio-visual representation, while the users explore the content through embodied interaction. The application contains three Kinect cameras for detecting the users and his/her movements and three video screens for presenting visual content. Six users were able to interact with the system at the same time.  The installation was exhibited in Vuores Housing Fair. The Fair took place in Vuores, located nearby Tampere. Our installation was available there for testing between July 13 and August 12, 2012 (Figure 2). During the one-month length piloting period circa 200 users tested the application and provided feedback of their experiences using the system. The participants were encouraged to experiment with the system and afterwards researchers conducted a survey on the use experience.

Energy Land Technologies

Figure 2. Energy Land Installation Space: up and running

The installation is rather complex from the technological point of view. The system contains three Kinect cameras for detecting the users.  Also, the sound system exploits 5 speakers for producing surround audio as well as 9 directional Panphonics [2] speakers for producing speech synthesis sounds.

Aesthetics as a Guide for Designing Sonic and Musical Interaction

When designing for aesthetic experiences, we approached the concept from the point of view of pragmatist aesthetics. Drawing from Dewey’s pragmatist aesthetics, the aesthetic experience emerges in the making process and the intended design is continuously under study [1]. Further, according to the pragmatist aesthetics view aesthetic experience emerges in a shared process between two groups: the designers and the end-users.

The initial idea for the interaction and sound design was to provide a sense of serendipity while also allowing a certain amount of control through sonic and generative music feedback for the user who interacted with the system. In line with Dewey, the way of working basing on pragmatist aesthetics allows for freedom of exploration and makes room for serendipity. Thus, the rigorous control of the end result is not an aim. In this research case it became clear, that developing generative behaviors affect implicitly the outcome of the aesthetic qualities of the audio and thus how the design process is in a continues state of flux. Later I will write more about the software architecture and its aesthetic principles in detail.

Game Audio Engine

The audio programming was realized with Pure Data software. Inspired by the patch cables of analog synthesizers, a PD program is composed of input and output patches that control the flow of data. I really like the open-ended nature of the visual programming. I find it a convenient way to explore generative strategies and to study computational aesthetics. However, in my experience it is easy to program oneself into a corner as the developed system gets larger and more complex. Thus, creating proper design documents in the planning phase is vital for the realization phase. For example, in the early phase of the sound design I divided the sounds into the following categories and mapped them to a spatial mixing map (figure 3).

Spatial Sound Map

Figure 3. Energy Game: audio setup and spatial sound map

The resulting sound scape has two themes: rainy and sunny atmospheres. In the system, these two generative music themes are altered continuously through various generative processes, and thus variations of the themes are continuously changing. In this way the sound scape avoids the common pitfall of people feeling dull after hearing the sound scape repetitively. I personally find this an important aspect when designing sounds for public spaces. That is the beauty of generative design strategies that they create unpredictable results over and over again.

You may listen the recording of the generative sound design from the player below. Please note that this version is rendered to stereo from original 5.1 mix down. I will also try to upload the program version of the outcome later for reference.

Energy Land Theme:

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

The Energy Land research case is still ongoing and we have two more installations to come. In particular, I am exited of our plans to combine ambient light with sound design. Based on the preliminary results, the generative strategies have a lot to offer when designing for aesthetic experiences and they could be applied to various contexts. My intention is to document the design and evaluation phases thoroughly and to publish the outcomes in articles later on with my colleagues. I will write more about the results later this year.

EDIT (August 14, 2013) : the resulting design and the outcomes of the project has been documented in detail in a conference paper. You may find the paper from the Publications page.

References

[1] Dewey, J. (1934) Art as Experience.
[2] Panphonics. Web site,  www.panphonics.fi (read in May 2, 2013)

Speak Your Mind

*

  • RSS
  • LinkedIn
  • Twitter
  • GitHub
  • Academia.edu
  • ResearchGate