[Industry] Predicting Customer Satisfaction with Soft Labels for Ordinal Classification
Etienne Manderscheid, Matthias Lee
Industry: Industry Industry Paper
Session 5: Industry (Poster)
Conference Room: Frontenac Ballroom and Queen's Quay
Conference Time: July 11, 16:15-17:45 (EDT) (America/Toronto)
Global Time: July 11, Session 5 (20:15-21:45 UTC)
TLDR:
In a typical call center, only up to 8\% of callers
leave a Customer Satisfaction (CSAT) survey
response at the end of the call, and these tend to
be customers with strongly positive or negative
experiences. To manage this data sparsity and
response bias, we outline a predictive CSAT
deep learning a...
You can open the
#paper-I188
channel in a separate window.
Abstract:
In a typical call center, only up to 8\% of callers
leave a Customer Satisfaction (CSAT) survey
response at the end of the call, and these tend to
be customers with strongly positive or negative
experiences. To manage this data sparsity and
response bias, we outline a predictive CSAT
deep learning algorithm that infers CSAT on
the 1-5 scale on inbound calls to the call center
with minimal latency. The key metric to maximize is the precision for CSAT = 1 (lowest
CSAT). We maximize this metric in two ways.
First, reframing the problem
as a binary class, rather than five-class problem during model fine-tuning, and then mapping binary outcomes back to five classes using
temperature-scaled model probabilities. Second, using soft labels to represent the classes.
The
result is a production model able to support key
customer workflows with high accuracy over
millions of calls a month.