Skip to search boxSkip to navigationSkip to main content

System building cost vs. output quality in data-to-text generation

  • Anja Belz
    ,
  • Eric Kow
Research output:
Chapter in Book/Conference proceeding with ISSN or ISBN
Conference contribution with ISSN or ISBN
Research
Peer-review

About the research

Data-to-text generation systems tend tobe knowledge-based and manually built,which limits their reusability and makes them time and cost-intensive to createand maintain. Methods for automating(part of) the system building process exist,but do such methods risk a loss inoutput quality? In this paper, we investigatethe cost/quality trade-off in generation system building. We comparefour new data-to-text systems which were created by predominantly automatic techniques against six existing systems for the same domain which were created by predominantlymanual techniques. We evaluate the ten systems using intrinsic automatic metrics and human quality ratings.We find that increasing the degree towhich system building is automated doesnot necessarily result in a reduction in outputquality. We find furthermore that standardautomatic evaluation metrics underestimate the quality of handcrafted systems and over-estimate the quality of automatically created systems.

Research output information

Output type

Research output:
Chapter in Book/Conference proceeding with ISSN or ISBN
Conference contribution with ISSN or ISBN
Research
Peer-review

Original language

English

Pages from-to (Number of pages)

Pages 16-24 (9 pages)

Publication milestones

  • Published - 30/03/2009

Publication status

Published - 30/03/2009

Place of publication

Athens, Greece

Publisher

Technografia Digital Press

Host publication title

Proceedings of the 12th European workshop on natural language generation (ENLG 2009)