## Abstract

Graphical models, such as Bayesian networks and Markov random fields (MRFs), represent statistical dependencies of variables by a graph. The max-product "belief propagation" algorithm is a local-message-passing algorithm on this graph that is known to converge to a unique fixed point when the graph is a tree. Furthermore, when the graph is a tree, the assignment based on the fixed point yields the most probable values of the unobserved variables given the observed ones. Recently, good empirical performance has been obtained by running the max-product algorithm (or the equivalent min-sum algorithm) on graphs with loops, for applications including the decoding of "turbo" codes. Except for two simple graphs (cycle codes and single-loop graphs) there has been little theoretical understanding of the max-product algorithm on graphs with loops. Here we prove a result on the fixed points of max-product on a graph with arbitrary topology and with arbitrary probability distributions (discrete- or continuous-valued nodes). We show that the assignment based on a fixed point is a "neighborhood maximum" of the posterior probability: the posterior probability of the max-product assignment is guaranteed to be greater than all other assignments in a particular large region around that assignment. The region includes all assignments that differ from the max-product assignment in any subset of nodes that form no more than a single loop in the graph. In some graphs, this neighborhood is exponentially large. We illustrate the analysis with examples.

Original language | English |
---|---|

Pages (from-to) | 736-744 |

Number of pages | 9 |

Journal | IEEE Transactions on Information Theory |

Volume | 47 |

Issue number | 2 |

DOIs | |

State | Published - Feb 2001 |

Externally published | Yes |

### Bibliographical note

Funding Information:Manuscript received January 3, 2000. This work was supported under Grants MURI-ARO-DAAH04-96-1-0341, MURI N00014-00-1-0637, and NSF IIS-9988642 Y. Weiss is with the Computer Science Division, University of California at Berkeley, Berkeley, CA 94720-1776 USA (e-mail: [email protected]). W. T. Freeman is with MERL, Mitsubishi Electric Research Labs., Cambridge, MA 02139 USA (e-mail: [email protected]). Communicated by B. J. Frey, Guest Editor. Publisher Item Identifier S 0018-9448(01)00724-6.

## Keywords

- Bayesian networks
- Belief propagations
- Markov random fields (MRFs)
- Max-product
- Maximum a posteriori (MAP) estimate
- Min-sum