Low-Resource Formality Controlled NMT Using Pre-trained LM
Priyesh Vakharia, Shree Vignesh S, Pranjali Basmatkar
The 20th International Conference on Spoken Language Translation Long Paper
TLDR:
This paper describes the UCSC's submission to the shared task on formality control for spoken language translation at IWSLT 2023. For this task, we explored the use of 'additive style intervention' using a pre-trained multilingual translation model, namely mBART. Compared to prior approaches where a
You can open the
#paper-IWSLT_37
channel in a separate window.
Abstract:
This paper describes the UCSC's submission to the shared task on formality control for spoken language translation at IWSLT 2023. For this task, we explored the use of 'additive style intervention' using a pre-trained multilingual translation model, namely mBART. Compared to prior approaches where a single style-vector was added to all tokens in the encoder output, we explored an alternative approach in which we learn a unique style-vector for each input token. We believe this approach, which we call 'style embedding intervention,' is better suited for formality control as it can potentially learn which specific input tokens to modify during decoding. While the proposed approach obtained similar performance to 'additive style intervention' for the supervised English-to-Vietnamese task, it performed significantly better for English-to-Korean, in which it achieved an average matched accuracy of 90.6 compared to 85.2 for the baseline. When we constrained the model further to only perform style intervention on the <bos> (beginning of sentence) token, the average matched accuracy improved further to 92.0, indicating that the model could learn to control the formality of the translation output based solely on the embedding of the <bos> token.