Enhancing the Transformer Decoder with Transition-based Syntax

Leshem Choshen, Omri Abend

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Notwithstanding recent advances, syntactic generalization remains a challenge for text decoders. While some studies showed gains from incorporating source-side symbolic syntactic and semantic structure into text generation Transformers, very little work addressed the decoding of such structure. We propose a general approach for tree decoding using a transition-based approach. Examining the challenging test case of incorporating Universal Dependencies syntax into machine translation, we present substantial improvements on test sets that focus on syntactic generalization, while presenting improved or comparable performance on standard MT benchmarks. Further qualitative analysis addresses cases where syntactic generalization in the vanilla Transformer decoder is inadequate and demonstrates the advantages afforded by integrating syntactic information.

Original languageAmerican English
Title of host publicationCoNLL 2022 - 26th Conference on Computational Natural Language Learning, Proceedings of the Conference
PublisherAssociation for Computational Linguistics (ACL)
Pages384-404
Number of pages21
ISBN (Electronic)9781959429074
StatePublished - 2022
Event26th Conference on Computational Natural Language Learning, CoNLL 2022 collocated and co-organized with EMNLP 2022 - Abu Dhabi, United Arab Emirates
Duration: 7 Dec 20228 Dec 2022

Publication series

NameCoNLL 2022 - 26th Conference on Computational Natural Language Learning, Proceedings of the Conference

Conference

Conference26th Conference on Computational Natural Language Learning, CoNLL 2022 collocated and co-organized with EMNLP 2022
Country/TerritoryUnited Arab Emirates
CityAbu Dhabi
Period7/12/228/12/22

Bibliographical note

Publisher Copyright:
©2022 Association for Computational Linguistics.

Fingerprint

Dive into the research topics of 'Enhancing the Transformer Decoder with Transition-based Syntax'. Together they form a unique fingerprint.

Cite this