System building cost vs. output quality in data-to-text generation
- Anja Belz,
- Eric Kow
Research output:
Chapter in Book/Conference proceeding with ISSN or ISBN
Conference contribution with ISSN or ISBN
Research
Peer-reviewAbout the research
Data-to-text generation systems tend tobe knowledge-based and manually built,which limits their reusability and makes them time and cost-intensive to createand maintain. Methods for automating(part of) the system building process exist,but do such methods risk a loss inoutput quality? In this paper, we investigatethe cost/quality trade-off in generation system building. We comparefour new data-to-text systems which were created by predominantly automatic techniques against six existing systems for the same domain which were created by predominantlymanual techniques. We evaluate the ten systems using intrinsic automatic metrics and human quality ratings.We find that increasing the degree towhich system building is automated doesnot necessarily result in a reduction in outputquality. We find furthermore that standardautomatic evaluation metrics underestimate the quality of handcrafted systems and over-estimate the quality of automatically created systems.
Documents and links
Final published version
License:Unspecified
Research output information
Output type
Research output:
Chapter in Book/Conference proceeding with ISSN or ISBN
Conference contribution with ISSN or ISBN
Research
Peer-reviewOriginal language
EnglishPages from-to (Number of pages)
Pages 16-24 (9 pages)Publication milestones
- Published - 30/03/2009
Publication status
Published - 30/03/2009
