Visible Cohesion: A Comparison of Reference Tracking in Sign, Speech, and Co-Speech Gesture

Pamela Perniss, Aslı Ozyurek

Research output: Contribution to journalArticle

Abstract

Establishing and maintaining reference is a crucial part of discourse. In spoken languages, differ- ential linguistic devices mark referents occurring in different referential contexts, that is, introduc- tion, maintenance, and re-introduction contexts. Speakers using gestures as well as users of sign languages have also been shown to mark referents differentially depending on the referential context. This article investigates the modality-specific contribution of the visual modality in marking referential context by providing a direct comparison between sign language (German Sign Language; DGS) and co-speech gesture with speech (German) in elicited narratives. Across all forms of expression, we find that referents in subject position are referred to with more marking material in re-introduction contexts compared to maintenance contexts. Furthermore, we find that spatial modification is used as a modality-specific strategy in both DGS and German co-speech gesture, and that the configuration of referent locations in sign space and gesture space corresponds in an iconic and consistent way to the locations of referents in the narrated event. However, we find that spatial modification is used in different ways for marking re-introduction and maintenance contexts in DGS and German co-speech gesture. The findings are discussed in relation to the unique contribution of the visual modality to reference tracking in discourse when it is used in a unimodal system with full linguistic structure (i.e., as in sign) versus in a bimodal system that is a composite of speech and ges- ture.
Original languageEnglish
Pages (from-to)36-60
Number of pages25
JournalTopics in Cognitive Science
Volume7
Issue number1
DOIs
Publication statusPublished - 31 Jan 2015

Bibliographical note

This is the peer reviewed version of the following article: Perniss, P. and Özyürek, A. (2015), Visible Cohesion: A Comparison of Reference Tracking in Sign, Speech, and Co-Speech Gesture. Topics in Cognitive Science, 7: 36–60, which has been published in final form at http://onlinelibrary.wiley.com/doi/10.1111/tops.12122/abstract. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving.

Keywords

  • Reference tracking
  • Co-speech gesture
  • Sign language
  • Visual modality
  • Use of space
  • Pointing

Fingerprint Dive into the research topics of 'Visible Cohesion: A Comparison of Reference Tracking in Sign, Speech, and Co-Speech Gesture'. Together they form a unique fingerprint.

  • Cite this