Discrete vs. continuous rating scales for language evaluation in NLP

Anja Belz, Eric Kow

Research output: Chapter in Book/Conference proceeding with ISSN or ISBNConference contribution with ISSN or ISBN

Abstract

Studies assessing rating scales are very common in psychology and related fields, but are rare in NLP. In this paper we assess discrete and continuous scales used for measuring quality assessments of computer-generated language. We conducted six separate experiments designed to investigate the validity, reliability, stability, interchangeability and sensitivity of discrete vs. continuous scales. We show that continuous scales are viable for use in language evaluation, and offer distinct advantages over discrete scales.
Original languageEnglish
Title of host publicationThe 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies
Place of PublicationStroudsburg, PA, USA
PublisherAssociation for Computational Linguistics
Pages230-235
Number of pages6
ISBN (Print)9781932432886
Publication statusPublished - 1 Jan 2011
EventThe 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Portland, Oregon, USA, 19-24 June, 2011
Duration: 1 Jan 2011 → …

Conference

ConferenceThe 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies
Period1/01/11 → …

Fingerprint Dive into the research topics of 'Discrete vs. continuous rating scales for language evaluation in NLP'. Together they form a unique fingerprint.

  • Cite this

    Belz, A., & Kow, E. (2011). Discrete vs. continuous rating scales for language evaluation in NLP. In The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (pp. 230-235). Association for Computational Linguistics. http://www.aclweb.org/anthology-new/P/P11/P11-1000.pdf