The Second Multilingual Surface Realisation Shared Task (SR'19): Overview and Evaluation Results

Simon Mille; Anja Belz; Bernd Bohnet; Yvette Graham; Leo Wanner

doi:10.18653/v1/D19-6301

The Second Multilingual Surface Realisation Shared Task (SR'19): Overview and Evaluation Results

Simon Mille
, Anja Belz
, Bernd Bohnet
, Yvette Graham
, Leo Wanner

Research output: Chapter in Book/Conference proceeding with ISSN or ISBN › Conference contribution with ISSN or ISBN

Abstract

We report results from the SR'19 Shared Task, the second edition of a multilingual surface realisation task organised as part of the EMNLP'19 Workshop on Multilingual Surface Realisation. As in SR'18, the shared task comprised two tracks with different levels of complexity: (a) a shallow track where the inputs were full UD structures with word order information removed and tokens lemmatised; and (b) a deep track where additionally, functional words and morphological information were removed. The shallow track was offered in eleven, and the deep track in three languages. Systems were evaluated (a) automatically, using a range of intrinsic metrics, and (b) by human judges in terms of readability and meaning similarity. This report presents the evaluation results, along with descriptions of the SR'19 tracks, data and evaluation methods. For full descriptions of the participating systems, please see the separate system reports elsewhere in this volume.

Original language	English
Title of host publication	Proceedings of the 2nd Workshop on Multilingual Surface Realisation (MSR 2019)
Place of Publication	Hong Kong, China
Publisher	Association for Computational Linguistics
Pages	1-17
Number of pages	17
DOIs	https://doi.org/10.18653/v1/D19-6301
Publication status	Published - 1 Nov 2019

Bibliographical note

ACL materials are Copyright © 1963–2019 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License.

Access to Document

10.18653/v1/D19-6301Licence: CC BY

The Second Multilingual Surface Realisation Shared Task (SR’19)Final published version, 381 KBLicence: CC BY

Cite this

@inproceedings{50b7d3e21f074aac965ebfe1ad026e0a,

title = "The Second Multilingual Surface Realisation Shared Task (SR'19): Overview and Evaluation Results",

abstract = "We report results from the SR'19 Shared Task, the second edition of a multilingual surface realisation task organised as part of the EMNLP'19 Workshop on Multilingual Surface Realisation. As in SR'18, the shared task comprised two tracks with different levels of complexity: (a) a shallow track where the inputs were full UD structures with word order information removed and tokens lemmatised; and (b) a deep track where additionally, functional words and morphological information were removed. The shallow track was offered in eleven, and the deep track in three languages. Systems were evaluated (a) automatically, using a range of intrinsic metrics, and (b) by human judges in terms of readability and meaning similarity. This report presents the evaluation results, along with descriptions of the SR'19 tracks, data and evaluation methods. For full descriptions of the participating systems, please see the separate system reports elsewhere in this volume.",

author = "Simon Mille and Anja Belz and Bernd Bohnet and Yvette Graham and Leo Wanner",

note = "ACL materials are Copyright {\textcopyright} 1963–2019 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License.",

year = "2019",

month = nov,

day = "1",

doi = "10.18653/v1/D19-6301",

language = "English",

pages = "1--17",

booktitle = "Proceedings of the 2nd Workshop on Multilingual Surface Realisation (MSR 2019)",

publisher = "Association for Computational Linguistics",

}

The Second Multilingual Surface Realisation Shared Task (SR'19): Overview and Evaluation Results. / Mille, Simon; Belz, Anja; Bohnet, Bernd et al.
Proceedings of the 2nd Workshop on Multilingual Surface Realisation (MSR 2019). Hong Kong, China: Association for Computational Linguistics, 2019. p. 1-17.

Research output: Chapter in Book/Conference proceeding with ISSN or ISBN › Conference contribution with ISSN or ISBN

TY - GEN

T1 - The Second Multilingual Surface Realisation Shared Task (SR'19)

T2 - Overview and Evaluation Results

AU - Mille, Simon

AU - Belz, Anja

AU - Bohnet, Bernd

AU - Graham, Yvette

AU - Wanner, Leo

N1 - ACL materials are Copyright © 1963–2019 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License.

PY - 2019/11/1

Y1 - 2019/11/1

N2 - We report results from the SR'19 Shared Task, the second edition of a multilingual surface realisation task organised as part of the EMNLP'19 Workshop on Multilingual Surface Realisation. As in SR'18, the shared task comprised two tracks with different levels of complexity: (a) a shallow track where the inputs were full UD structures with word order information removed and tokens lemmatised; and (b) a deep track where additionally, functional words and morphological information were removed. The shallow track was offered in eleven, and the deep track in three languages. Systems were evaluated (a) automatically, using a range of intrinsic metrics, and (b) by human judges in terms of readability and meaning similarity. This report presents the evaluation results, along with descriptions of the SR'19 tracks, data and evaluation methods. For full descriptions of the participating systems, please see the separate system reports elsewhere in this volume.

AB - We report results from the SR'19 Shared Task, the second edition of a multilingual surface realisation task organised as part of the EMNLP'19 Workshop on Multilingual Surface Realisation. As in SR'18, the shared task comprised two tracks with different levels of complexity: (a) a shallow track where the inputs were full UD structures with word order information removed and tokens lemmatised; and (b) a deep track where additionally, functional words and morphological information were removed. The shallow track was offered in eleven, and the deep track in three languages. Systems were evaluated (a) automatically, using a range of intrinsic metrics, and (b) by human judges in terms of readability and meaning similarity. This report presents the evaluation results, along with descriptions of the SR'19 tracks, data and evaluation methods. For full descriptions of the participating systems, please see the separate system reports elsewhere in this volume.

U2 - 10.18653/v1/D19-6301

DO - 10.18653/v1/D19-6301

M3 - Conference contribution with ISSN or ISBN

SP - 1

EP - 17

BT - Proceedings of the 2nd Workshop on Multilingual Surface Realisation (MSR 2019)

PB - Association for Computational Linguistics

CY - Hong Kong, China

ER -

The Second Multilingual Surface Realisation Shared Task (SR'19): Overview and Evaluation Results

Abstract

Bibliographical note

Access to Document

Fingerprint

Cite this