Abstract
Current measures for evaluating text simplification systems focus on evaluating lexical text aspects, neglecting its structural aspects. In this paper we propose the first measure to address structural aspects of text simplification, called SAMSA. It leverages recent advances in semantic parsing to assess simplification quality by decomposing the input based on its semantic structure and comparing it to the output. SAMSA provides a reference-less automatic evaluation procedure, avoiding the problems that reference-based methods face due to the vast space of valid simplifications for a given sentence. Our human evaluation experiments show both SAMSA's substantial correlation with human judgments, as well as the deficiency of existing reference-based measures in evaluating structural simplification.1.
| Original language | English |
|---|---|
| Title of host publication | Long Papers |
| Publisher | Association for Computational Linguistics (ACL) |
| Pages | 685-696 |
| Number of pages | 12 |
| ISBN (Electronic) | 9781948087278 |
| State | Published - 2018 |
| Event | 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018 - New Orleans, United States Duration: 1 Jun 2018 → 6 Jun 2018 |
Publication series
| Name | NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference |
|---|---|
| Volume | 1 |
Conference
| Conference | 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018 |
|---|---|
| Country/Territory | United States |
| City | New Orleans |
| Period | 1/06/18 → 6/06/18 |
Bibliographical note
Publisher Copyright:© 2018 The Association for Computational Linguistics.