Cross-language differences in how voice quality and f0 contours map to affect

Abstract

The relationship between prosody and perceived affect involves multiple variables. This paper explores the interplay of three: voice quality, f0 contour, and the hearer's language background. Perception tests were conducted with speakers of Irish English, Russian, Spanish, and Japanese using three types of synthetic stimuli: (1) stimuli varied in voice quality, (2) stimuli of uniform (modal) voice quality incorporating affect-related f0 contours, and (3) stimuli combining specific non-modal voice qualities with the affect-related f0 contours of (2). The participants rated the stimuli for the presence/strength of affective colouring on six bipolar scales, e.g., happy-sad. The results suggest that stimuli incorporating non-modal voice qualities, with or without f0 variation, are generally more effective in affect cueing than stimuli varying only in f0. Along with similarities in the affective responses across these languages, many points of divergence were found, both in terms of the range and strength of affective responses overall and in terms of specific stimulus-to-affect associations. The f0 contour may play a more important role, and tense voice a lesser role in affect signalling in Japanese and Spanish than in Irish English and Russian. The greatest cross-language differences emerged for the affects intimate, formal, stressed, and relaxed.