Neural correlates of episodic memory buildup in naturalistic viewing behavior

In everyday life, we bind visual samples of the world into a coherent episodic memory via eye movements. Previous eye tracking research revealed that visual scrutiny, gaze transitions between event elements, and refixations during encoding predict successful memory formation. However, little is known about the neural mechanisms of these processes. Here, we combine EEG and eye tracking to investigate the buildup of episodic memory representations in naturalistic viewing behavior.

Twenty-eight participants encoded a series of events. Each event consisted of elements from three categories (faces, places, objects) and was presented for 10 s. After a distractor task, memory was tested for all event-specific element combinations. To succeed in the test, participants had to separate the different events by making strong associations between within-event elements. The simultaneously recorded EEG and eye movements were analyzed at the encoding stage of the task.

A major problem of EEG-eye movement coregistration in free viewing concerns the overlapping effects of sequential saccades on EEG. We overcame this by using the regression-based deconvolution method, which allows correction of such overlap as well as removal of the confounding effects from eye movement characteristics and ordering factors, such as fixation rank. After deconvolution, we extracted EEG theta and alpha activity in epochs from -200 to +400 ms relative to the fixation onset.

As expected, subsequent memory performance for the whole event increased with the cumulative number of fixations within single elements and inter-category gaze transitions between elements. We found that these two gaze memory effects are associated with two distinct memory effects of the fixation-related EEG activity: theta synchronization over the frontal and centro-parietal areas, respectively. The frontal theta effect may indicate the sampling of individual elements, whereas the centro-parietal theta effect may reflect the binding of elements into a coherent episodic event. Furthermore, memory performance was predicted by gaze returns to already visited categories. This refixation effect was associated with a fixation-related alpha desynchronization over the occipital areas. This may indicate the pivotal role of refixations in temporal accumulation of visual information that is needed for building a coherent representation of the whole event.

We conclude that the combination of theta synchronization and alpha desynchronization at the fixation level supports the buildup of a coherent episodic memory across sequential eye movements.