FLamE: Few-shot Learning from Natural Language Explanations
Yangqiaoyu Zhou, Yiming Zhang, Chenhao Tan
Main: Interpretability and Analysis of Models for NLP Main-oral Paper
Session 5: Interpretability and Analysis of Models for NLP (Oral)
Conference Room: Metropolitan East
Conference Time: July 11, 16:15-17:15 (EDT) (America/Toronto)
Global Time: July 11, Session 5 (20:15-21:15 UTC)
Keywords:
free-text/natural language explanations
TLDR:
Natural language explanations have the potential to provide rich information that in principle guides model reasoning.
Yet, recent work by Lampinen et al. has shown limited utility of natural language explanations in improving classification.
To effectively learn from explanations, we present FLamE,...
You can open the
#paper-P3332
channel in a separate window.
Abstract:
Natural language explanations have the potential to provide rich information that in principle guides model reasoning.
Yet, recent work by Lampinen et al. has shown limited utility of natural language explanations in improving classification.
To effectively learn from explanations, we present FLamE, a two-stage few-shot learning framework that first generates explanations using GPT-3, and then fine-tunes a smaller model (e.g., RoBERTa) with generated explanations.
Our experiments on natural language inference demonstrate effectiveness over strong baselines,
increasing accuracy by 17.6\% over GPT-3 Babbage and 5.7\% over GPT-3 Davinci in e-SNLI.
Despite improving classification performance, human evaluation surprisingly reveals that the majority of generated explanations does not adequately justify classification decisions.
Additional analyses point to the important role of label-specific cues (e.g., ``not know'' for the neutral label) in generated explanations.