Counterfactual Probing for the Influence of Affect and Specificity on Intergroup Bias

Venkata Subrahmanyan Govindarajan, David I Beaver, Kyle Mahowald, Junyi Jessy Li

📝 Paper Anthology Underline 🪧 Poster 📺 Watch Video on Underline Add to Favorites

Findings: Computational Social Science and Cultural Analytics Findings Paper

Session 1: Computational Social Science and Cultural Analytics (Virtual Poster)
Conference Room: Pier 7&8
Conference Time: July 10, 11:00-12:30 (EDT) (America/Toronto)
Global Time: July 10, Session 1 (15:00-16:30 UTC)
Spotlight Session: Spotlight - Metropolitan East (Spotlight)
Conference Room: Metropolitan East
Conference Time: July 10, 19:00-21:00 (EDT) (America/Toronto)
Global Time: July 10, Spotlight Session (23:00-01:00 UTC)
Keywords: nlp tools for social analysis, quantiative analyses of news and/or social media
TLDR: While existing work on studying bias in NLP focues on negative or pejorative language use, Govindarajan et al. (2023) offer a revised framing of bias in terms of intergroup social context, and its effects on language behavior. In this paper, we investigate if two pragmatic features (specificity and ...
You can open the #paper-P2462 channel in a separate window.
Abstract: While existing work on studying bias in NLP focues on negative or pejorative language use, Govindarajan et al. (2023) offer a revised framing of bias in terms of intergroup social context, and its effects on language behavior. In this paper, we investigate if two pragmatic features (specificity and affect) systematically vary in different intergroup contexts --- thus connecting this new framing of bias to language output. Preliminary analysis finds modest correlations between specificity and affect of tweets with supervised intergroup relationship (IGR) labels. Counterfactual probing further reveals that while neural models finetuned for predicting IGR reliably use affect in classification, the model's usage of specificity is inconclusive.