COST-SID Workshop on Sonification, 12-15-June at T-Labs in Berlin.
The topics investigated were:
- Aesthetics in Sonifications of Elite Rowing Data
- Sonification of Spatial Gestures on Mobile Phones
- Sonic Spirit Level
- Sonification in Support of Shape Feedback
During this workshop I led a group working on the case study of aesthetics in the sonification of rowing. The case study was provided by Nina Schaffert who was awarded the Best Poster Prize at ICAD 09.
Schaffert N. Mattes K. and Effenberg A.O. (2009), A Sound Design for the Purposes of Movement Optimisation in Elite Sport (Using the Example of Rowing), Proceedings of the 15th International Conference on Auditory Display, Copenhagen, Denmark, May 18-22, 2009. Download
We produced a range of different sonifications that explored theories and approaches to sonic information design.
Sin-ification – (Reiner Gerhet, Nina Schaffert)
This video shows the pilot study decribed in Nina’s poster in which a sine tone is used to sonify the acceleration of the skiff. From the video it is very clear that there is more information about rowing in the sonified data than meets the eye. The athletes and the coaches also found this sonification to be very promising.
The minimisation of the forward and back reversals, heard as the oars enter and leave the water, would improve the technique of the rowers and the speed of the boat.
The target users are elite rowers who may have expectations of pleasantness, enjoyability and musicalness of the sonification. For this reason our working group chose to explore approaches to the aesthetic dimensions of sonifications of the rowing dataset.
Xylophone MIDI-fication – (Anton Schertenlieb, Nina Schaffert, Stephen Barrass)
Would it make a difference to hear the sonification played on a more familiar musical instrument ? In this example the data is played on a Xylophone. This sonification was made by loading the data into the xSonify tool in order to map it to a MIDI Xylophone. It could just as easily be a piano, harpsichord, trombone or any other instrument. Does the choice of instrument effect the acceptance, understanding or perception of the data in any way ? The Xylophone was chosen because it has a short envelope duration suited to rapid playback. Since we were not able to render the MIDI to an audio file it was played out the headphone and recorded with Audacity on another laptop (please excuse some background chatter that was also recorded during the process !)
Ear-cleaning Sonification (Hanna Buhl and Georg Spehr)
Acoustic Ecology is a movement that seeks to encourage the appreciation and preservation of natural soundscapes. Ear-cleaning is a technique developed in acoustic ecology for learning to attend and observe the soundscape. This attention to the sonic environment may be very helpful in sports activities such as rowing where the eyes are already busy. In experiments it was found that when rowers wore ear-plugs their performance was reduced. The skiff is full of sounds produced by movement through the water, the oars, the rowlocks, the seats and the physical exertions of the rowers. The rowers also listen to the other skiffs in a race. This raises the possibility that the rowers could be taught to listen to auditory cues about synchronicity, rowing motion, and acceleration that are natural consequence of rowing.
Here is a recording made inside a rowing skiff.
Ecological/Metaphorical Sonification (John Williamson, Roderick Murray-Smith)
The soundscape in a rowing skiff includes a range of natural sounds that can provide a basis for auditory metaphors in a sonification design. This approach is similar to the Auditory Icons based on everyday sounds that have been developed for auditory interfaces. In this sonification computational models of splashing, wooden and metal impacts communicate the context is rowing. Impacts on these models are triggered by events in the data such as turning points and thresholds in the acceleration and the rate of change of acceleration. The sonification was produced in a unified Python programming environment with python(x,y) for data processing and pysndobj for sound synthesis and sequencing.
Techno Music Probe (Stephen Barrass)
This is an exploration of the concept of music-alisation in the techno genre. The software Albatat Muse was used to generate the soundtrack from movements in the video. This is a probe, rather than a prototype. The only link between the soundtrack and the video is the syncing of the music generated at 152 bpm with the movement of the oars at 38 (x 4 = 152) strokes per minute. The changes in the pitch corresponding with the drive and recovery phases, together with the sync of the percussion, raise interesting possibilities for a musically based sonification of the data.
Music-ification prototype (Stephen Barrass)
This is a prototype inspired by the Techno-music probe. The sonification was produced by loading the data into Sonification Sandbox. This first required the data to be adjusted with Excel so that all values were positive. Once the data was mapped to MIDI the output file was then loaded into Garageband. This file was then rendered using the FM Percussive synth, and the Metal Pad. Unfortuately the tempo of the MIDI data was not saved in the file transfer of the MIDI file from Sandbox, and Garageband only allows 240 bpm which is not fast enough. This meant the rendered audio had to be tempo adjusted 12x using Audacity. A narrative element was introduced by adjusting the mix over the course so that a change in the music conveys the final push to the finish.
PAF Streaming (Reiner Gerhet and Stephen Barrass)
This sonification is an extension of the PD patch used to produce the original Sin-ification. The oscillator was replaced with a Phase Aligned Formant (PAF) synthesiser. The acceleration maps to the pitch (as before) and the derivative of acceleration maps to both the width and centroid of a formant. This causes moments of rapid change in acceleration (or jerking) to emerge as a distinctive figure against the smoother bass ground. This is an example of Stream-based Sonification in which auditory figure / grounds emerge from relations in data, rather than from pre-determined top down “channels”.
Through the workshop we learned of the work of Chris Henkelmann who has been exploring ways to improve the aesthetic quality of realtime sporting feedback.
You can find his examples of PAF, tristimulus and other synthesis techniques applied to rowing data in the examples.zip at the bottom of this page.
The variation in these examples demonstrate the immensity of the space of sonifications that are possible for the same dataset. A blank canvas and the broad palette of synthesis algorithms raises the question of how to go about designing a sonification. The sonification design needs to balance functionality and aesthetics in a brief that includes users, tasks, and data, and must also consider learnability, context, aesthetics, emotive aspects. This is where a comprehensive method for designing sonifications can guide the string of decisions at many different levels that are required for a sonification that works in contexts beyond the scientific laboratory or the media arts gallery.
Sonic Branding (Hanna Buhl, Georg Spehr, Stephen Barrass)
There is also reseach on the sonification of rowing in other countries, such as Sweden. We discussed the possibility that the sonification could also include some reference to the national identity of the rowing team. Could the national anthem be part of the sonic branding ? We look forward to the Sonified Olympics where rowing teams from Germany, Sweden and Australia have sonifications in their training regime. May the best sonification win ;)
SONIFICATION.COM 
Comments are closed.