On Guaranteed Optimal Robust Explanations for NLP Models

Nicola Paoletti, Department of Informatics at King's College London, UK

  • 18.10.2022, 15:00 - 16:00

A seminar with Nicola Paoletti, Department of Informatics at King's College London, UK, on guaranteed Optimal robust explanations for NLP models. October 18 at 3pm-4pm in room M104 in Reykjavik University. 

Computing systems pervade every aspect of our daily lives and of society. Moreover, they increasingly make autonomous decisions that affect their users, such as whether a patient should receive some medication, when and how much, or whether we are eligible for a bank loan. Safety and reliability are therefore primary concerns for the developers and users of those systems. Some of Nicola Paoletti's research deals with data-driven analysis of computing systems and aims at providing guarantees that they behave as expected and that their decisions are explainable and interpretable. In this talk at Reykjavik University, he will describe some of his work on computing explanations for the decisions taken by neural network models that underlie much of the most successful systems used in natural language processing today. 

Abstract: We build on abduction-based explanations for machine learning and develop a method for computing local explanations for neural network models in natural language processing (NLP). Our explanations comprise a subset of the words of the input text that satisfies two key features: optimality w.r.t. a user-defined cost function, such as the length of explanation, and robustness, in that they ensure prediction invariance for any bounded perturbation in the embedding space of the left-out words. 

We present two solution algorithms, respectively based on implicit hitting sets and maximum universal subsets, introducing a number of algorithmic improvements to speed up convergence of hard instances. We show how our method can be configured with different perturbation sets in the embedded space and used to detect bias in predictions by enforcing include/exclude constraints on biased terms, as well as to enhance existing heuristic-based NLP explanation frameworks such as Anchors. 

We evaluate our framework on three widely-used sentiment analysis tasks and texts of up to 100 words from SST, Twitter and IMDB datasets, demonstrating the effectiveness of the derived explanations. Joint work with Agnieszka Zbrzezny (University of Warmia and Mazury), Emanuele La Malfa, Rhiannon Michelmore, and Marta Kwiatkowska (University of Oxford). Appeared in IJCAI 2021. See https://arxiv.org/pdf/2105.03640.pdf for the paper on which the talk is based. Short bio: Nicola is a Senior Lecturer in the Department of Informatics at King's College London. In the past four years, he has been a Lecturer at the Department of Computer Science at Royal Holloway, University of London. Previously, he has been a post-doc at Stony Brook University (USA) and University of Oxford, after an internship at Microsoft Research Cambridge (UK). He obtained his Ph.D. in Information Sciences and Complex Systems from the Universita' di Camerino (Italy). 

Nicola's interests are in safety and security assurance of cyber-physical (aka autonomous) systems, or CPSs, with an emphasis on biomedical applications. His research aims to develop formal analysis methods (verification, control, and synthesis) to design CPSs that are provably correct. With CPSs increasingly incorporating machine-learning components for e.g., sensing, control and model predictions, his work also focuses on data-driven verification of CPSs, whereby formal analysis and principled learning methods come together to provide correctness guarantees and interpretability, while accounting for the uncertainty and (potential) brittleness introduced by the learning components. https://nicolapaoletti.com/


Vinsamlegast athugið að á viðburðum Háskólans í Reykjavík (HR) eru teknar ljósmyndir og myndbönd sem notuð eru í markaðsstarfi HR. Hægt er að nálgast frekari upplýsingar á ru.is eða með því að senda tölvupóst á netfangið: personuvernd@ru.is
Please note that at events hosted at Reykjavik University (RU), photographs and videos are taken which might be used for RU marketing purposes. Read more about this on our ru.is or send an e-mail: personuvernd@ru.is