Synesthetic Associations Between Voice and Gestures in Preverbal Infants: Weak Effects and Methodological Concerns

Adult humans spontaneously associate visual features, such as size and direction of movement, with phonetic properties like vowel quality and auditory pitch. A number of recent studies have claimed that looking time in preverbal infants reveals the same associations, which would indicate that some cross-modal correspondences are the result of perceptual biases. Here we tested 30 infants of age 7-13 months, who were exposed to pairs of audiovisual stimuli presented first sequentially and then side by side. The stimuli consisted of a visual object (computer-animated ball or filmed human hand) moving sinusoidally, vertically, or in a U-shape and accompanied by a sliding voice-like tone. Sequential presentation revealed no preference for either audiovisual synchrony or synesthetic congruency, while side-by-side presentation revealed a small preference for incongruent stimuli. The effect of congruency was similar for the animated ball and filmed human hand. These findings extend the results of previous research on pitch-motion synesthesia in preverbal infants, which used animations and sliding whistles, to more ecologically relevant stimuli such as voice and gestures. If infants and adults share the same preferences for non-arbitrary mappings between manual gestures and intonation, this could indicate that cross-modal correspondences facilitate language acquisition. On the other hand, a critical survey of the field revealed that previous studies of audiovisual cross-modal correspondences in infants suffer from replication failures due to poor robustness of the reported effect with respect to experimental stimuli and testing procedure. We therefore argue that the research on cross-modal correspondences in infants would profit from using alternative testing methods in addition to preferential looking and call for replication of previously reported congruency effects.