Generalized prioritized sweeping

David Andre, Nir Friedman, Ronald Parr

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

45 Scopus citations

Abstract

Prioritized sweeping is a model-based reinforcement learning method that attempts to focus an agent's limited computational resources to achieve a good estimate of the value of environment states. To choose effectively where to spend a costly planning step, classic prioritized sweeping uses a simple heuristic to focus computation on the states that are likely to have the largest errors. In this paper, we introduce generalized prioritized sweeping, a principled method for generating such estimates in a representation-specific manner. This allows us to extend prioritized sweeping beyond an explicit, state-based representation to deal with compact representations that are necessary for dealing with large state spaces. We apply this method for generalized model approximators (such as Bayesian networks), and describe preliminary experiments that compare our approach with classical prioritized sweeping.

Original languageAmerican English
Title of host publicationAdvances in Neural Information Processing Systems 10 - Proceedings of the 1997 Conference, NIPS 1997
PublisherNeural information processing systems foundation
Pages1001-1007
Number of pages7
ISBN (Print)0262100762, 9780262100762
StatePublished - 1998
Externally publishedYes
Event11th Annual Conference on Neural Information Processing Systems, NIPS 1997 - Denver, CO, United States
Duration: 1 Dec 19976 Dec 1997

Publication series

NameAdvances in Neural Information Processing Systems
ISSN (Print)1049-5258

Conference

Conference11th Annual Conference on Neural Information Processing Systems, NIPS 1997
Country/TerritoryUnited States
CityDenver, CO
Period1/12/976/12/97

Fingerprint

Dive into the research topics of 'Generalized prioritized sweeping'. Together they form a unique fingerprint.

Cite this