Construction of bilingual multimodal corpora of referring expressions in collaborative problem solving

Takenobu Tokunaga, Ryu Iida, Masaaki Yasuhara, Asuka Terai, David Morris, Anja Belz

Research output: Chapter in Book/Conference proceeding with ISSN or ISBNConference contribution with ISSN or ISBN

Abstract

This paper presents on-going work on constructing bilingual multimodal corpora of referring expressions in collaborative problem solving for English and Japanese. The corpora were collected from dialogues in which two participants collaboratively solved Tangram puzzles with a puzzle simulator. Extra-linguistic information such as operations on puzzle pieces, mouse cursor position and piece positions were recorded in synchronisation with utterances. The speech data was transcribed and time-aligned with the extra-linguistic information. Referring expressions in utterances that refer to puzzle pieces were annotated in terms of their spans, their referents and their other attributes. The Japanese corpus has already been completed, but the English counter-part is still undergoing annotation. We have conducted a preliminary comparative analysis of both corpora, mainly with respect to task completion time, task success rates and attributes of referring expressions. These corpora showed significant differences in task completion time and success rate.
Original languageEnglish
Title of host publication8th Workshop on Asian Language Resources
Place of PublicationMichigan
PublisherUniversity of Michigan
Pages38-46
Number of pages9
Publication statusPublished - 1 Aug 2010
Event8th Workshop on Asian Language Resources - Coling
Duration: 1 Aug 2010 → …

Workshop

Workshop8th Workshop on Asian Language Resources
Period1/08/10 → …

Fingerprint Dive into the research topics of 'Construction of bilingual multimodal corpora of referring expressions in collaborative problem solving'. Together they form a unique fingerprint.

  • Cite this

    Tokunaga, T., Iida, R., Yasuhara, M., Terai, A., Morris, D., & Belz, A. (2010). Construction of bilingual multimodal corpora of referring expressions in collaborative problem solving. In 8th Workshop on Asian Language Resources (pp. 38-46). University of Michigan. http://clair.eecs.umich.edu/aan/paper.php?paper_id=W10-3206