Transition-Independent Decentralized Markov Decision Processes

Raphen Becker; Shlomo Zilberstein; Victor Lesser; Claudia V. Goldman

doi:10.1145/860575.860583

Transition-Independent Decentralized Markov Decision Processes

Raphen Becker^*
, Shlomo Zilberstein
, Victor Lesser
, Claudia V. Goldman

^*Corresponding author for this work

Research output: Contribution to conference › Paper › peer-review

81 Scopus citations

Abstract

There has been substantial progress with formal models for sequential decision making by individual agents using the Markov decision process (MDP). However, similar treatment of multi-agent systems is lacking. A recent complexity result, showing that solving decentralized MDPs is NEXP-hard, provides a partial explanation. To overcome this complexity barrier, we identify a general class of transition-independent decentralized MDPs that is widely applicable. The class consists of independent collaborating agents that are tied together through a global reward function that depends upon both of their histories. We present a novel algorithm for solving this class of problems and examine its properties. The result is the first effective technique to solve optimally a class of decentralized MDPs. This lays the foundation for further work in this area on both exact and approximate solutions.

Original language	English
Pages	41-48
Number of pages	8
DOIs	https://doi.org/10.1145/860575.860583
State	Published - 2003
Externally published	Yes
Event	Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 03 - Melbourne, Vic., Australia Duration: 14 Jul 2003 → 18 Jul 2003

Conference

Conference	Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 03
Country/Territory	Australia
City	Melbourne, Vic.
Period	14/07/03 → 18/07/03

Keywords

Decentralized MDP
Decision-theoretic planning

Access to Document

10.1145/860575.860583

Cite this

@conference{504a8fe57d85428ba8793039418ac673,

title = "Transition-Independent Decentralized Markov Decision Processes",

abstract = "There has been substantial progress with formal models for sequential decision making by individual agents using the Markov decision process (MDP). However, similar treatment of multi-agent systems is lacking. A recent complexity result, showing that solving decentralized MDPs is NEXP-hard, provides a partial explanation. To overcome this complexity barrier, we identify a general class of transition-independent decentralized MDPs that is widely applicable. The class consists of independent collaborating agents that are tied together through a global reward function that depends upon both of their histories. We present a novel algorithm for solving this class of problems and examine its properties. The result is the first effective technique to solve optimally a class of decentralized MDPs. This lays the foundation for further work in this area on both exact and approximate solutions.",

keywords = "Decentralized MDP, Decision-theoretic planning",

author = "Raphen Becker and Shlomo Zilberstein and Victor Lesser and Goldman, \{Claudia V.\}",

year = "2003",

doi = "10.1145/860575.860583",

language = "אנגלית",

pages = "41--48",

note = "Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 03 ; Conference date: 14-07-2003 Through 18-07-2003",

}

TY - CONF

T1 - Transition-Independent Decentralized Markov Decision Processes

AU - Becker, Raphen

AU - Zilberstein, Shlomo

AU - Lesser, Victor

AU - Goldman, Claudia V.

PY - 2003

Y1 - 2003

N2 - There has been substantial progress with formal models for sequential decision making by individual agents using the Markov decision process (MDP). However, similar treatment of multi-agent systems is lacking. A recent complexity result, showing that solving decentralized MDPs is NEXP-hard, provides a partial explanation. To overcome this complexity barrier, we identify a general class of transition-independent decentralized MDPs that is widely applicable. The class consists of independent collaborating agents that are tied together through a global reward function that depends upon both of their histories. We present a novel algorithm for solving this class of problems and examine its properties. The result is the first effective technique to solve optimally a class of decentralized MDPs. This lays the foundation for further work in this area on both exact and approximate solutions.

AB - There has been substantial progress with formal models for sequential decision making by individual agents using the Markov decision process (MDP). However, similar treatment of multi-agent systems is lacking. A recent complexity result, showing that solving decentralized MDPs is NEXP-hard, provides a partial explanation. To overcome this complexity barrier, we identify a general class of transition-independent decentralized MDPs that is widely applicable. The class consists of independent collaborating agents that are tied together through a global reward function that depends upon both of their histories. We present a novel algorithm for solving this class of problems and examine its properties. The result is the first effective technique to solve optimally a class of decentralized MDPs. This lays the foundation for further work in this area on both exact and approximate solutions.

KW - Decentralized MDP

KW - Decision-theoretic planning

UR - https://www.scopus.com/pages/publications/1142293055

U2 - 10.1145/860575.860583

DO - 10.1145/860575.860583

M3 - ???researchoutput.researchoutputtypes.contributiontoconference.paper???

AN - SCOPUS:1142293055

SP - 41

EP - 48

T2 - Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 03

Y2 - 14 July 2003 through 18 July 2003

ER -

Transition-Independent Decentralized Markov Decision Processes

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this