Event Segmentation in Audio Description Reception: Impacts on Media Access, Inclusion, and Computer-Generated Description

Audio description (AD) serves a critical role in making audio-visual media accessible to visually impaired audiences, aiming to enhance their viewing experience and comprehension (Holsanova 2016, 2022). A central inquiry in the context of successful AD pertains to how the sequence of events unfolding in a film aligns with the verbal narration conveyed through AD (Vandaele 2012). Accumulating research demonstrates considerable agreement among individuals in discerning distinct and meaningful event units in films and recognising event boundaries, usually occurring at shifts in spatiotemporal configurations (Radvansky & Zacks, 2017; Zacks, 2013). To evaluate the efficacy of AD during a film experience, it is necessary to understand how the AD influences the perception and segmentation of the film’s unfolding chain-of-events (Holsanova, Blomberg, Blomberg, Gärdenfors & Johansson 2023).

In the present study, we use an event segmentation task as a method to assess the comprehension of film narratives (Johansson, Rastegar, Lyberg–Åhlander & Holsanova 2024). Sighted participants watched a Swedish film, while visually impaired participants experienced the same film with two AD versions — one explicitly expressing key event boundaries and another containing more implicitly conveyed ones. Both sighted and visually impaired participants were asked to delineate the narrative unfolding into distinct meaningful events.

Our findings indicate that visually impaired participants perceived event boundaries similarly to sighted participants, suggesting that AD effectively conveys the event structure. However, in the AD version with implicit expressions, event boundaries were less likely to be recognized. These results shed light on event segmentation dynamics in films, emphasizing the importance of how event boundaries are presented in AD. This has significant implications for improving the cinematic experience for visually impaired viewers, emphasizing the need for clear, explicit information about event boundaries within AD.

Event segmentation also holds significance in the domain of automated computer-generated description, since algorithms need to be trained to identify actions within dynamic scenes, discern connections between frames and actions, recognize event boundaries, and effectively express them. Our results are thus equally essential for automated computer-generated video description (Braun, Starr & Laaksonen, 2020; Starr, Braun & Delfani, 2020). A proper comprehension of how visually impaired end users perceive event boundaries in audio description can significantly contribute to the development of algorithms capable of accurately tracking referents across narrative depictions of time and space. This, in turn, may offer valuable assistance in establishing nominal and pronominal continuity and coherence, thereby addressing some of the key challenges inherent in the performance of such algorithms.

Acknowledgement
The research has been conducted together with Roger Johansson, Tina Rastegar and Viveka Lyberg–Åhlander. We gratefully acknowledge Lund University Humanities Laboratory, Tina Weidelt for her thorough and professional work with providing and narrating the audio descriptions, and Linn Petersdotter for her persistent and careful work with data collection.
This work was supported by a grant from FORTE 2018-00200 (Swedish Research Council for Health, Working Life and Welfare) and by TSI 2019 Lund University grant “Audio Description and Accessible Information” (2021-2025).