Audio description and multimodality: Accessing meaning-making in popular scientific texts

Activity: Talk or presentationPresentation


Title Audio description and multimodality: Accessing meaning-making in popular scientific texts
Person and role
Description The paper focuses on the production and reception processes in audio description of a Swedish multimodal popular scientific journal Forskning och Framsteg (Research and Progress). The contents of the journal are made accessible for blind and visually impaired audiences by producing an audio version. When interpreting the printed version and transforming it into an audio version, the processes of reception and production coincide. The meaning-making processes are uncovered with the help of a think aloud protocol.In my analysis, I apply the framework of socio-semiotic theories (Kress & Van Leeuwen 1996/2006, O’Halloran et al., 2012, Jewitt 2014), in particular on visual construction of specialized knowledge (Unsworth 1997, Unsworth & Cleirigh 2014), in combination with cognitive theories on reception of multimodality (Holsanova 2008, Holsanova 2014a,b). First, the printed journal is analysed in accordance with Unsworth (1997), focusing on how the resources of text, images and graphics are deployed in scientific explanation and how the meaning is constructed by the visuals. Second, the interpretative process of meaning-making is uncovered by think aloud protocols. In order to produce an aural version of the complex text, the interpreter must assess what to describe, how to describe it, and when to describe it (Holsanova 2015). He combines the contents of the available resources, makes judgements about relevant information, ways of verbalizing it, fills in the gaps missing in the interplay of the resources and re-arranges the order of information for optimal flow and understanding. In this way, he contributes to multimodal literacy (Walsh 2010, Kress & Jewitt 2003). The aural version of the journal is finally compared to the printed version to show how the semiotic interplay has been realized for the end users.

2018 Aug 15

Research areas and keywords


Related organisations
2018 Aug 15

Event (Conference)

Abbrev. TitleICOM
LocationUniversity of Southern Denmark, Odense
Degree of recognitionInternational event

Related projects

Jana Holsanova


Project: Research

Jana Holsanova & Cecilia Wadensjö

2010/01/15 → …

Project: Network

View all (3)