The GREC main subject reference generation challenge 2009: Overview and evaluation results

Anja Belz; Eric Kow; Sebastian Varges

The GREC main subject reference generation challenge 2009: Overview and evaluation results

Anja Belz, Eric Kow, Sebastian Varges

University of Brighton

Research output: Chapter in Book/Conference proceeding with ISSN or ISBN › Conference contribution with ISSN or ISBN › peer-review

Abstract

The GREC-MSR Task at Generation Challenges 2009 required participating systems to select coreference chains to the main subject of short encyclopaedic texts collected from Wikipedia. Three teams submitted one system each, and we additionally created four baseline systems. Systems were tested automatically using existing intrinsic metrics. We also evaluated systems extrinsically by applying coreference resolution tools to the outputs and measuring the success of the tools. In addition,systems were tested in an intrinsic evaluation involving human judges. This report describes the GREC-MSR Task and the evaluation methods applied, gives brief descriptions of the participating systems, and presents the evaluation results.

Original language	English
Title of host publication	Proceedings of the 2009 Workshop on Language Generation and Summarisation (UCNLG+Sum 2009)
Publication status	Published - 2009
Event	2009 Workshop on Language Generation and Summarisation - Suntec, Singapore Duration: 1 Jan 2009 → …

Workshop

Workshop	2009 Workshop on Language Generation and Summarisation
Period	1/01/09 → …

Access to Document

http://aclweb.org/anthology-new/W/W09/W09-2816.pdfLicence: Unspecified

Cite this

@inproceedings{07881cb22c78442286a090e47cac7612,

title = "The GREC main subject reference generation challenge 2009: Overview and evaluation results",

abstract = "The GREC-MSR Task at Generation Challenges 2009 required participating systems to select coreference chains to the main subject of short encyclopaedic texts collected from Wikipedia. Three teams submitted one system each, and we additionally created four baseline systems. Systems were tested automatically using existing intrinsic metrics. We also evaluated systems extrinsically by applying coreference resolution tools to the outputs and measuring the success of the tools. In addition,systems were tested in an intrinsic evaluation involving human judges. This report describes the GREC-MSR Task and the evaluation methods applied, gives brief descriptions of the participating systems, and presents the evaluation results.",

author = "Anja Belz and Eric Kow and Sebastian Varges",

year = "2009",

language = "English",

booktitle = "Proceedings of the 2009 Workshop on Language Generation and Summarisation (UCNLG+Sum 2009)",

note = "2009 Workshop on Language Generation and Summarisation ; Conference date: 01-01-2009",

}

TY - GEN

T1 - The GREC main subject reference generation challenge 2009: Overview and evaluation results

AU - Belz, Anja

AU - Kow, Eric

AU - Varges, Sebastian

PY - 2009

Y1 - 2009

N2 - The GREC-MSR Task at Generation Challenges 2009 required participating systems to select coreference chains to the main subject of short encyclopaedic texts collected from Wikipedia. Three teams submitted one system each, and we additionally created four baseline systems. Systems were tested automatically using existing intrinsic metrics. We also evaluated systems extrinsically by applying coreference resolution tools to the outputs and measuring the success of the tools. In addition,systems were tested in an intrinsic evaluation involving human judges. This report describes the GREC-MSR Task and the evaluation methods applied, gives brief descriptions of the participating systems, and presents the evaluation results.

AB - The GREC-MSR Task at Generation Challenges 2009 required participating systems to select coreference chains to the main subject of short encyclopaedic texts collected from Wikipedia. Three teams submitted one system each, and we additionally created four baseline systems. Systems were tested automatically using existing intrinsic metrics. We also evaluated systems extrinsically by applying coreference resolution tools to the outputs and measuring the success of the tools. In addition,systems were tested in an intrinsic evaluation involving human judges. This report describes the GREC-MSR Task and the evaluation methods applied, gives brief descriptions of the participating systems, and presents the evaluation results.

M3 - Conference contribution with ISSN or ISBN

BT - Proceedings of the 2009 Workshop on Language Generation and Summarisation (UCNLG+Sum 2009)

T2 - 2009 Workshop on Language Generation and Summarisation

Y2 - 1 January 2009

ER -