Shared-task evaluations in HLT: lessons for NLG

Anja Belz, Adam Kilgarriff

Research output: Chapter in Book/Conference proceeding with ISSN or ISBNConference contribution with ISSN or ISBN

Abstract

While natural language generation (NLG) has a strong evaluation tradition, in particular in userbased and task-oriented evaluation, it has never evaluated different approaches and techniques by comparing their performance on the same tasks (shared-task evaluation, STE). NLG is characterised by a lack of consolidation of results, and by isolation from the rest of NLP where STE is now standard. It is, moreover, a shrinking field (state-of-the-art MT and summarisation no longer perform generation as a subtask) which lacks the kind of funding and participation that natural language understanding (NLU) has attracted.
Original languageEnglish
Title of host publicationProceedings of the 4th International Conference on Natural Language Generation (INLG'06)
Place of PublicationGermany
PublisherDBLP
Pages133-135
Number of pages3
Publication statusPublished - 1 Jan 2006
EventProceedings of the 4th International Conference on Natural Language Generation (INLG'06) - Sydney, Australia
Duration: 1 Jan 2006 → …

Conference

ConferenceProceedings of the 4th International Conference on Natural Language Generation (INLG'06)
Period1/01/06 → …

Fingerprint

Consolidation

Keywords

  • Natural language generation

Cite this

Belz, A., & Kilgarriff, A. (2006). Shared-task evaluations in HLT: lessons for NLG. In Proceedings of the 4th International Conference on Natural Language Generation (INLG'06) (pp. 133-135). Germany: DBLP.
Belz, Anja ; Kilgarriff, Adam. / Shared-task evaluations in HLT: lessons for NLG. Proceedings of the 4th International Conference on Natural Language Generation (INLG'06). Germany : DBLP, 2006. pp. 133-135
@inproceedings{cec6961817be434db822d09dc7cb7d47,
title = "Shared-task evaluations in HLT: lessons for NLG",
abstract = "While natural language generation (NLG) has a strong evaluation tradition, in particular in userbased and task-oriented evaluation, it has never evaluated different approaches and techniques by comparing their performance on the same tasks (shared-task evaluation, STE). NLG is characterised by a lack of consolidation of results, and by isolation from the rest of NLP where STE is now standard. It is, moreover, a shrinking field (state-of-the-art MT and summarisation no longer perform generation as a subtask) which lacks the kind of funding and participation that natural language understanding (NLU) has attracted.",
keywords = "Natural language generation",
author = "Anja Belz and Adam Kilgarriff",
year = "2006",
month = "1",
day = "1",
language = "English",
pages = "133--135",
booktitle = "Proceedings of the 4th International Conference on Natural Language Generation (INLG'06)",
publisher = "DBLP",

}

Belz, A & Kilgarriff, A 2006, Shared-task evaluations in HLT: lessons for NLG. in Proceedings of the 4th International Conference on Natural Language Generation (INLG'06). DBLP, Germany, pp. 133-135, Proceedings of the 4th International Conference on Natural Language Generation (INLG'06), 1/01/06.

Shared-task evaluations in HLT: lessons for NLG. / Belz, Anja; Kilgarriff, Adam.

Proceedings of the 4th International Conference on Natural Language Generation (INLG'06). Germany : DBLP, 2006. p. 133-135.

Research output: Chapter in Book/Conference proceeding with ISSN or ISBNConference contribution with ISSN or ISBN

TY - GEN

T1 - Shared-task evaluations in HLT: lessons for NLG

AU - Belz, Anja

AU - Kilgarriff, Adam

PY - 2006/1/1

Y1 - 2006/1/1

N2 - While natural language generation (NLG) has a strong evaluation tradition, in particular in userbased and task-oriented evaluation, it has never evaluated different approaches and techniques by comparing their performance on the same tasks (shared-task evaluation, STE). NLG is characterised by a lack of consolidation of results, and by isolation from the rest of NLP where STE is now standard. It is, moreover, a shrinking field (state-of-the-art MT and summarisation no longer perform generation as a subtask) which lacks the kind of funding and participation that natural language understanding (NLU) has attracted.

AB - While natural language generation (NLG) has a strong evaluation tradition, in particular in userbased and task-oriented evaluation, it has never evaluated different approaches and techniques by comparing their performance on the same tasks (shared-task evaluation, STE). NLG is characterised by a lack of consolidation of results, and by isolation from the rest of NLP where STE is now standard. It is, moreover, a shrinking field (state-of-the-art MT and summarisation no longer perform generation as a subtask) which lacks the kind of funding and participation that natural language understanding (NLU) has attracted.

KW - Natural language generation

M3 - Conference contribution with ISSN or ISBN

SP - 133

EP - 135

BT - Proceedings of the 4th International Conference on Natural Language Generation (INLG'06)

PB - DBLP

CY - Germany

ER -

Belz A, Kilgarriff A. Shared-task evaluations in HLT: lessons for NLG. In Proceedings of the 4th International Conference on Natural Language Generation (INLG'06). Germany: DBLP. 2006. p. 133-135