SustaiNLP
Organizers: Nafise Sadat Moosavi, Iryna Gurevych, Yufang Hou, Gyuwan Kim, Young Jin Kim, Tal Schuster, Ameeta Agrawal
The Natural Language Processing (NLP) community has, in recent years, demonstrated a notable focus on improving higher scores on standard benchmarks and taking the lead on community-wide leaderboards (e.g., GLUE, SentEval). While this aspiration has led to improvements in benchmark performance of (predominantly neural) models, it has also came at a cost, i.e., increased model complexity and the ever-growing amount of computational resources required for training and using the current state-of-the-art models. Moreover, the recent research efforts have, for the most part, failed to identify sources of empirical gains in models, often failing to empirically justify the model complexity beyond benchmark performance. \newline Because of these easily observable trends, we have proposed the SustaiNLP workshop with the goal of promoting more sustainable NLP research and practices, with two main objectives: (1) encouraging development of more efficient NLP models; and (2) providing simpler architectures and empirical justification of model complexity. For both aspects, we will encourage submissions from all topical areas of NLP.
You can open
the
#workshop-SustaiNLP
channel in separate windows.
Workshop Papers
ADEPT: Adapter-based Efficient Prompt Tuning Approach for Language Models
Authors: Aditya Shah, Surendrabikram Thapa, Aneesh Jain, Lifu Huang
Go to Paper
NLU on Data Diets: Dynamic Data Subset Selection for NLP Classification Tasks
Authors: Jean-michel Attendu, Jean-philippe Corbeil
Go to Paper
On the Interactions of Structural Constraints and Data Resources for Structured Prediction
Authors: Zhisong Zhang, Emma Strubell, Eduard Hovy
Go to Paper
Can we Pretrain a SotA Legal Language Model on a Budget From Scratch?
Authors: Joel Niklaus, Daniele Giofre
Go to Paper
Is a Video worth n n Images? A Highly Efficient Approach to Transformer-based Video Question Answering
Authors: Chenyang Lyu, Tianbo Ji, Yvette Graham, Jennifer Foster
Go to Paper
KwikBucks: Correlation Clustering with Cheap-Weak and Expensive-Strong Signals
Authors: Sandeep Silwal, Sara Ahmadian, Andrew Nystrom, Andrew Mccallum, Deepak Ramachandran, Mehran Kazemi
Go to Paper
How to Unleash the Power of Large Language Models for Few-shot Relation Extraction?
Authors: Xin Xu, Yuqi Zhu, Xiaohan Wang, Ningyu Zhang
Go to Paper
Prompting language models improves performance in imbalanced setting
Authors: Jay Mohta
Go to Paper
KGQA Without Retraining
Authors: Nick Mckenna, Priyanka Sen
Go to Paper
MANER: Mask Augmented Named Entity Recognition for Extreme Low-Resource Languages
Authors: Shashank Sonkar, Zichao Wang, Richard Baraniuk
Go to Paper
Efficient and Interpretable Compressive Text Summarisation with Unsupervised Dual-Agent Reinforcement Learning
Authors: Peggy Tang, Junbin Gao, Lei Zhang, Zhiyong Wang
Go to Paper
Adder Encoder for Pre-trained Language Model
Authors: Jianbang Ding, Suiyun Zhang, Linlin Li
Go to Paper
Semantic-Oriented Unlabeled Priming for Large-Scale Language Models
Authors: Yanchen Liu, Timo Schick, Hinrich Schtze
Go to Paper
Exploring the Effect of Frequency Resolution in FNet
Authors: Gregory Szumel, Ghazal Khalighinejad, Rickard Stureborg, Sam Wiseman
Go to Paper
Towards Adaptable and Interactive Image Captioning with Data Augmentation and Episodic Memory
Authors: Aliki Anagnostopoulou, Mareike Hartmann, Daniel Sonntag
Go to Paper
Sequence Reducible Holdout Loss for Language Model Pretraining
Authors: Raghuveer Thirukovalluru, Bhuwan Dhingra, Sam Wiseman
Go to Paper
Corpus Complexity Matters in Pretraining Language Models
Authors: Ameeta Agrawal, Suresh Singh
Go to Paper
PersonaPKT: Building Personalized Dialogue Agents via Parameter-efficient Knowledge Transfer
Authors: Xu Han, Bin Guo, Yoon Jung, Benjamin Yao, Yu Zhang, Xiaohu Liu, Chenlei Guo
Go to Paper
oBERTa: Improving Sparse Transfer Learning via improved initialization, distillation, and pruning regimes
Authors: Daniel Campos, Alexandre Marques, Mark Kurtz, Cheng Xiang Zhai
Go to Paper
Small Character Models Match Large Word Models for Autocomplete Under Memory Constraints
Authors: Ganesh Jawahar, Subhabrata Mukherjee, Debadeepta Dey, Muhammad Abdul-mageed, Laks Lakshmanan, V.s., Caio Mendes, Gustavo De Rosa, Shital Shah
Go to Paper
Query Encoder Distillation via Embedding Alignment is a Strong Baseline Method to Boost Dense Retriever Online Efficiency
Authors: Yuxuan Wang, Lyu Hong
Go to Paper
Minimalist Entity Disambiguation for Mid-Resource Languages
Authors: Benno Kruit
Go to Paper
Quick Dense Retrievers Consume KALE: Post Training KullbackLeibler Alignment of Embeddings for Asymmetrical dual encoders
Authors: Daniel Campos, Alessandro Magnani, Chengxiang Zhai
Go to Paper
Lessons on Parameter Sharing across Layers in Transformers
Authors: Sho Takase, Shun Kiyono
Go to Paper
To Asymmetry and Beyond: Structured Pruning of Sequence to Sequence Models for Improved Inference Efficiency
Authors: Daniel Campos, Chengxiang Zhai
Go to Paper
Small is the New Big: Pre-finetuned compact models are better for Asynchronous Active Learning