System building cost vs. output quality in data-to-text generation

Anja Belz, Eric Kow

Research output: Chapter in Book/Conference proceeding with ISSN or ISBNConference contribution with ISSN or ISBN

Abstract

Data-to-text generation systems tend tobe knowledge-based and manually built,which limits their reusability and makes them time and cost-intensive to createand maintain. Methods for automating(part of) the system building process exist,but do such methods risk a loss inoutput quality? In this paper, we investigatethe cost/quality trade-off in generation system building. We comparefour new data-to-text systems which were created by predominantly automatic techniques against six existing systems for the same domain which were created by predominantlymanual techniques. We evaluate the ten systems using intrinsic automatic metrics and human quality ratings.We find that increasing the degree towhich system building is automated doesnot necessarily result in a reduction in outputquality. We find furthermore that standardautomatic evaluation metrics underestimate the quality of handcrafted systems and over-estimate the quality of automatically created systems.
Original languageEnglish
Title of host publicationProceedings of the 12th European workshop on natural language generation (ENLG 2009)
Place of PublicationAthens, Greece
PublisherTechnografia Digital Press
Pages16-24
Number of pages9
Publication statusPublished - 30 Mar 2009
EventProceedings of the 12th European workshop on natural language generation (ENLG 2009) - Athens, Greece, 30-31 March, 2009
Duration: 30 Mar 2009 → …

Workshop

WorkshopProceedings of the 12th European workshop on natural language generation (ENLG 2009)
Period30/03/09 → …

Fingerprint Dive into the research topics of 'System building cost vs. output quality in data-to-text generation'. Together they form a unique fingerprint.

  • Cite this

    Belz, A., & Kow, E. (2009). System building cost vs. output quality in data-to-text generation. In Proceedings of the 12th European workshop on natural language generation (ENLG 2009) (pp. 16-24). Technografia Digital Press. http://www.itri.brighton.ac.uk/~Anja.Belz/Publications/belz-kow-enlg09.pdf