Reference-less measure of faithfulness for grammatical error correction

Leshem Choshen; Omri Abend

doi:10.18653/v1/n18-2020

Reference-less measure of faithfulness for grammatical error correction

Leshem Choshen
, Omri Abend

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

26 Scopus citations

Abstract

We propose USIM, a semantic measure for Grammatical Error Correction (GEC) that measures the semantic faithfulness of the output to the source, thereby complementing existing reference-less measures (RLMs) for measuring the output's grammaticality. USIM operates by comparing the semantic symbolic structure of the source and the correction, without relying on manually-curated references. Our experiments establish the validity of USIM, by showing that (1) semantic annotation can be consistently applied to ungrammatical text; (2) valid corrections obtain a high USIM similarity score to the source; and (3) invalid corrections obtain a lower score.

Original language	English
Title of host publication	Short Papers
Publisher	Association for Computational Linguistics (ACL)
Pages	124-129
Number of pages	6
ISBN (Electronic)	9781948087292
DOIs	https://doi.org/10.18653/v1/n18-2020
State	Published - 2018
Event	2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018 - New Orleans, United States Duration: 1 Jun 2018 → 6 Jun 2018

Publication series

Name	NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference
Volume	2

Conference

Conference	2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018
Country/Territory	United States
City	New Orleans
Period	1/06/18 → 6/06/18

Bibliographical note

Publisher Copyright:
© 2018 Association for Computational Linguistics.

Access to Document

10.18653/v1/n18-2020

Cite this

Choshen, L., & Abend, O. (2018). Reference-less measure of faithfulness for grammatical error correction. In Short Papers (pp. 124-129). (NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference; Vol. 2). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/n18-2020

@inproceedings{244f6a0dcdfd401b95ceca1634cf19aa,

title = "Reference-less measure of faithfulness for grammatical error correction",

abstract = "We propose USIM, a semantic measure for Grammatical Error Correction (GEC) that measures the semantic faithfulness of the output to the source, thereby complementing existing reference-less measures (RLMs) for measuring the output's grammaticality. USIM operates by comparing the semantic symbolic structure of the source and the correction, without relying on manually-curated references. Our experiments establish the validity of USIM, by showing that (1) semantic annotation can be consistently applied to ungrammatical text; (2) valid corrections obtain a high USIM similarity score to the source; and (3) invalid corrections obtain a lower score.",

author = "Leshem Choshen and Omri Abend",

note = "Publisher Copyright: {\textcopyright} 2018 Association for Computational Linguistics.; 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018 ; Conference date: 01-06-2018 Through 06-06-2018",

year = "2018",

doi = "10.18653/v1/n18-2020",

language = "אנגלית",

series = "NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference",

publisher = "Association for Computational Linguistics (ACL)",

pages = "124--129",

booktitle = "Short Papers",

address = "ארצות הברית",

}

Choshen, L & Abend, O 2018, Reference-less measure of faithfulness for grammatical error correction. in Short Papers. NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, vol. 2, Association for Computational Linguistics (ACL), pp. 124-129, 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018, New Orleans, United States, 1/06/18. https://doi.org/10.18653/v1/n18-2020

Reference-less measure of faithfulness for grammatical error correction. / Choshen, Leshem; Abend, Omri.
Short Papers. Association for Computational Linguistics (ACL), 2018. p. 124-129 (NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference; Vol. 2).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Reference-less measure of faithfulness for grammatical error correction

AU - Choshen, Leshem

AU - Abend, Omri

PY - 2018

Y1 - 2018

N2 - We propose USIM, a semantic measure for Grammatical Error Correction (GEC) that measures the semantic faithfulness of the output to the source, thereby complementing existing reference-less measures (RLMs) for measuring the output's grammaticality. USIM operates by comparing the semantic symbolic structure of the source and the correction, without relying on manually-curated references. Our experiments establish the validity of USIM, by showing that (1) semantic annotation can be consistently applied to ungrammatical text; (2) valid corrections obtain a high USIM similarity score to the source; and (3) invalid corrections obtain a lower score.

AB - We propose USIM, a semantic measure for Grammatical Error Correction (GEC) that measures the semantic faithfulness of the output to the source, thereby complementing existing reference-less measures (RLMs) for measuring the output's grammaticality. USIM operates by comparing the semantic symbolic structure of the source and the correction, without relying on manually-curated references. Our experiments establish the validity of USIM, by showing that (1) semantic annotation can be consistently applied to ungrammatical text; (2) valid corrections obtain a high USIM similarity score to the source; and (3) invalid corrections obtain a lower score.

UR - https://www.scopus.com/pages/publications/85063106088

U2 - 10.18653/v1/n18-2020

DO - 10.18653/v1/n18-2020

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85063106088

T3 - NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference

SP - 124

EP - 129

BT - Short Papers

PB - Association for Computational Linguistics (ACL)

T2 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018

Y2 - 1 June 2018 through 6 June 2018

ER -

Choshen L, Abend O. Reference-less measure of faithfulness for grammatical error correction. In Short Papers. Association for Computational Linguistics (ACL). 2018. p. 124-129. (NAACL HLT 2018 - 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference). doi: 10.18653/v1/n18-2020

Reference-less measure of faithfulness for grammatical error correction

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this