We explore (1) how speakers employ prosodic and gestural cues to deal with ambiguous sentences and (2) what the audiovisual resolution reveals about communicative efficiency and effort. Thirty-two Chinese speakers articulated twenty-two ambiguous Mandarin sentences. Half could be semantically differentiated using prosody (half couldn’t). First, participants spontaneously articulated all sentences and explained the meaning to a confederate, revealing their dominant interpretations. Second, participants articulated the same ambiguous sentences twice according to hints suggesting two different meanings. Analyses of participants’ prosody and gestures showed that for sentences that can be prosodically distinguished, participants employed various prosodic cues, while 51.85% of sentences were accompanied by referential (iconic; pointing) gestures and 17.33% of sentences were accompanied by non-referential (beat; interactional) gestures. However, when prosodic cues cannot disambiguate, participants used more referential (97.30%) but fewer non-referential gestures (1.28%). In conclusion, speakers adopt a multimodal approach to achieve communicative efficiency with a trade-off between modalities.

Authors:

Jiajun GAO: University of Nottingham Ningbo China; Yan Gu: University of Essex