On the challenges of visualizing data with audio or “audiolization” ?

 

Central idea behind the episode:

The aspiration of our podcast is to examine and challenge the boundaries of the communication of data through audio. Visualisation has shown to be a useful method to convey information. The visualisation of data also allows to discover patterns or structures in large datasets that are otherwise hard or impossible  to read. Trying to communicate the same information verbally proves to be problematic. There is no formal technique or software in audiolization typically by comparison to data visualization. For example, in the case that a visualization is made based on location data, naturally a map would be used with a visualization. This raises questions on how a map would be presented by sound only. Is talking about visualization like dancing about architecture? When a visualization is being communicated as a verbal transcription the potential added value of audiolization is being overlooked. The capacity of audio to communicate information can be seen in a geiger counter or a metal detector. The difference in the proximity of sounds suggest a relation between the device and the quantity or distance from the object it is measuring. Alternatively, sounds can also be used to create a narrative. The visualisation through audio offers an alternative perspective allowing to communicate a story differently. An audio visualization published by Al Jazeera on Saudi air attacks on Yemen is a representative example of how sounds can offer a sensory experience on an additional level (Labs 2019). Cairo states that mechanisms of detecting basic features, also called preattentive features, aim at finding tricks and shortcuts for the brain to make sense of the information gathered from the senses. These preattentive features are applied on visual information. However, this information can be valuable for understanding and creating audiolization as well. The Berlin school introduces “Gestalt psychology” in relation to the interpretation of images (Wertheimer 2017; Koffka 2013; Köhler 1929). The concepts of connectedness on the area’s of figure/ground, proximity, similarity, continuation, law of prägnanz, closure, uniform destiny and uniform connectedness can be considered when creating a data sonification (Kaper, Wiebel, and Tipei 1999). The creation of a fore and background can be created in sound, for example by making a background score, presenting the main information louder or contrasting with the score. We used background sounds like airplanes, and coughs and sneezes representing the data. Proximity of sounds can indicate a relation between the sounds. Sounds that are placed near each other may indicate that they are related. Similarly, the other principles can be applied to sounds as well. This podcast is an experimental interactive research report on using the affordances of podcasting to present information through a different sensory experience. With this experiment we argue that we do not only “understand because we see’ (Cairo 2012) as Cairo states, but we also understand because we can here. We do this by presenting the monthly emissions of nitric dioxide in Amsterdam in the form of a song.  We are using music, which is mathematically structured.