Solving Sokoban with Forward-Backward Reinforcement Learning

Yaron Shoham, Gal Elidan

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

Despite seminal advances in reinforcement learning in recent years, many domains where the rewards are sparse, e.g. given only at task completion, remain quite challenging. In such cases, it can be beneficial to tackle the task both from its beginning and end, and make the two ends meet. Existing approaches that do so, however, are not effective in the common scenario where the strategy needed near the end goal is very different from the one that is effective earlier on. In this work we propose a novel RL approach for such settings. In short, we first train a backward-looking agent with a simple relaxed goal, and then augment the state representation of the forward-looking agent with straightforward hint features. This allows the learned forward agent to leverage information from backward plans, without mimicking their policy. We demonstrate the efficacy of our approach on the challenging game of Sokoban, where we substantially surpass learned solvers that generalize across levels, and are competitive with SOTA performance of the best highly-crafted systems. Impressively, we achieve these results while learning from a small number of practice levels and using simple RL techniques.

Original languageAmerican English
Title of host publication14th International Symposium on Combinatorial Search, SoCS 2021
EditorsHang Ma, Ivan Serina
PublisherAssociation for the Advancement of Artificial Intelligence
Pages191-193
Number of pages3
ISBN (Electronic)9781713834557
StatePublished - 2021
Externally publishedYes
Event14th International Symposium on Combinatorial Search, SoCS 2021 - Guangzhou, Virtual, China
Duration: 26 Jul 202130 Jul 2021

Publication series

Name14th International Symposium on Combinatorial Search, SoCS 2021

Conference

Conference14th International Symposium on Combinatorial Search, SoCS 2021
Country/TerritoryChina
CityGuangzhou, Virtual
Period26/07/2130/07/21

Bibliographical note

Publisher Copyright:
Copyright © 2021, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

Fingerprint

Dive into the research topics of 'Solving Sokoban with Forward-Backward Reinforcement Learning'. Together they form a unique fingerprint.

Cite this