[Demo] UINAUIL: A Unified Benchmark for Italian Natural Language Understanding
Valerio Basile, Livio Bioglio, Alessio Bosca, Cristina Bosco, Viviana Patti
Anthology
Underline
Demo: Resources and Evaluation (demo) Demo Paper
Demo Session 5: Resources and Evaluation (demo) (Poster)
Conference Room: Frontenac Ballroom and Queen's Quay
Conference Time: July 11, 16:15-17:45 (EDT) (America/Toronto)
Global Time: July 11, Demo Session 5 (20:15-21:45 UTC)
TLDR:
This paper introduces the Unified Interactive Natural Understanding of the Italian Language (UINAUIL), a benchmark of six tasks for Italian Natural Language Understanding. We present a description of the tasks and software library that collects the data from the European Language Grid, harmonizes th...
You can open the
#paper-D90
channel in a separate window.
Abstract:
This paper introduces the Unified Interactive Natural Understanding of the Italian Language (UINAUIL), a benchmark of six tasks for Italian Natural Language Understanding. We present a description of the tasks and software library that collects the data from the European Language Grid, harmonizes the data format, and exposes functionalities to facilitates data manipulation and the evaluation of custom models. We also present the results of tests conducted with available Italian and multilingual language models on UINAUIL, providing an updated picture of the current state of the art in Italian NLU.