Audience Research and Multimodality: What Eye Tracking Reveals about Newspaper Reading

On closer inspection, the so-called iconic turn in recent media history is a multimodal turn. Media communication has not only been enriched by pictures but has turned from a text-medium into a multi-medium, i.e. into a complex system of different modes like design, colours, pictures, graphics, and typography. Along with the multimodal turn, the linear structure of media communication has become non-linear, which confronts the audience with the problems of selection and attention management. Each recipient has to decide what she wants to receive, in which order and with what intensity. From the perspective of the producer, media communication is no longer only concerned with selecting and creating content. Nowadays, it must also include a compositional and navigational structure for the readers who meet various kinds of content on screens and pages. Thus, audience research under these new conditions cannot not only focus on media effects like knowledge acquisition or attitude change. In addition, it has to take a closer look at the selection processes and attention management. In newspaper research and usability research, this can be done by eye tracking studies.
In our presentation, we will evaluate the eye tracking methodology in regard to its functionality for investigating multimodal communication. We will show that the traditional eye tracking research follows the paradigm of a salience theory since it conceptualises reading as being controlled by salient elements of the design. In contrast, we will argue that an interactional theory, which integrates media features and audience factors, is more appropriate for explaining the reception of multimodal media discourse.