Correctness of belief propagation in Gaussian graphical models of arbitrary topology

Yair Weiss, William T. Freeman

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

37 Scopus citations

Abstract

Local "belief propagation" rules of the sort proposed by Pearl [15] are guaranteed to converge to the correct posterior probabilities in singly connected graphical models. Recently, a number of researchers have empirically demonstrated good performance of "loopy belief propagation"-using these same rules on graphs with loops. Perhaps the most dramatic instance is the near Shannon-limit performance of "Turbo codes", whose decoding algorithm is equivalent to loopy belief propagation. Except for the case of graphs with a single loop, there has been little theoretical understanding of the performance of loopy propagation. Here we analyze belief propagation in networks with arbitrary topologies when the nodes in the graph describe jointly Gaussian random variables. We give an analytical formula relating the true posterior probabilities with those calculated using loopy propagation. We give sufficient conditions for convergence and show that when belief propagation converges it gives the correct posterior means for all graph topologies, not just networks with a single loop. The related "max-product" belief propagation algorithm finds the maximum posterior probability estimate for singly connected networks. We show mat, even for non-Gaussian probability distributions, the convergence points of the max-product algorithm in loopy networks are maxima over a particular large local neighborhood of the posterior probability. These results help clarify the empirical performance results and motivate using the powerful belief propagation algorithm in a broader class of networks.

Original languageAmerican English
Title of host publicationAdvances in Neural Information Processing Systems 12 - Proceedings of the 1999 Conference, NIPS 1999
PublisherNeural information processing systems foundation
Pages673-679
Number of pages7
ISBN (Print)0262194503, 9780262194501
StatePublished - 2000
Externally publishedYes
Event13th Annual Neural Information Processing Systems Conference, NIPS 1999 - Denver, CO, United States
Duration: 29 Nov 19994 Dec 1999

Publication series

NameAdvances in Neural Information Processing Systems
ISSN (Print)1049-5258

Conference

Conference13th Annual Neural Information Processing Systems Conference, NIPS 1999
Country/TerritoryUnited States
CityDenver, CO
Period29/11/994/12/99

Fingerprint

Dive into the research topics of 'Correctness of belief propagation in Gaussian graphical models of arbitrary topology'. Together they form a unique fingerprint.

Cite this