Reinforcement Learning Explainability via Model Transforms (Student Abstract)

Mira Finkelstein, Lucy Liu, Yoav Kolumbus, David C. Parkes, Jeffrey S. Rosenshein, Sarah Keren

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Understanding the emerging behaviors of reinforcement learning agents may be difficult because such agents are often trained using highly complex and expressive models. In recent years, most approaches developed for explaining agent behaviors rely on domain knowledge or on an analysis of the agent's learned policy. For some domains, relevant knowledge may not be available or may be insufficient for producing meaningful explanations. We suggest using formal model abstractions and transforms, previously used mainly for expediting the search for optimal policies, to automatically explain discrepancies that may arise between the behavior of an agent and the behavior that is anticipated by an observer. We formally define this problem of Reinforcement Learning Policy Explanation (RLPE), suggest a class of transforms which can be used for explaining emergent behaviors, and suggest methods for searching efficiently for an explanation. We demonstrate the approach on standard benchmarks.

Original languageEnglish
Title of host publicationIAAI-22, EAAI-22, AAAI-22 Special Programs and Special Track, Student Papers and Demonstrations
PublisherAssociation for the Advancement of Artificial Intelligence
Pages12943-12944
Number of pages2
ISBN (Electronic)1577358767, 9781577358763
StatePublished - 30 Jun 2022
Event36th AAAI Conference on Artificial Intelligence, AAAI 2022 - Virtual, Online
Duration: 22 Feb 20221 Mar 2022

Publication series

NameProceedings of the 36th AAAI Conference on Artificial Intelligence, AAAI 2022
Volume36

Conference

Conference36th AAAI Conference on Artificial Intelligence, AAAI 2022
CityVirtual, Online
Period22/02/221/03/22

Bibliographical note

Publisher Copyright:
Copyright © 2022, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

Fingerprint

Dive into the research topics of 'Reinforcement Learning Explainability via Model Transforms (Student Abstract)'. Together they form a unique fingerprint.

Cite this