Learning arbitrary statistical mixtures of discrete distributions

Jian Li, Yuval Rabani, Leonard J. Schulman, Chaitanya Swamy

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

9 Scopus citations

Abstract

We study the problem of learning from unlabeled samples very general statistical mixture models on large finite sets. Specifically, the model to be learned, θ, is a probability distribution over probability distributions p, where each such p is a probability distribution over [n] = {1,2,...,n}. When we sample from θ, we do not observe p directly, but only indirectly and in very noisy fashion, by sampling from [n] repeatedly, independently K times from the distribution p. The problem is to infer θ to high accuracy in transportation (earthmover) distance. We give the first efficient algorithms for learning this mixture model without making any restricting assumptions on the structure of the distribution θ. We bound the quality of the solution as a function of the size of the samples K and the number of samples used. Our model and results have applications to a variety of unsupervised learning scenarios, including learning topic models and collaborative filtering.

Original languageAmerican English
Title of host publicationSTOC 2015 - Proceedings of the 2015 ACM Symposium on Theory of Computing
PublisherAssociation for Computing Machinery
Pages743-752
Number of pages10
ISBN (Electronic)9781450335362
DOIs
StatePublished - 14 Jun 2015
Event47th Annual ACM Symposium on Theory of Computing, STOC 2015 - Portland, United States
Duration: 14 Jun 201517 Jun 2015

Publication series

NameProceedings of the Annual ACM Symposium on Theory of Computing
Volume14-17-June-2015
ISSN (Print)0737-8017

Conference

Conference47th Annual ACM Symposium on Theory of Computing, STOC 2015
Country/TerritoryUnited States
CityPortland
Period14/06/1517/06/15

Bibliographical note

Publisher Copyright:
© Copyright 2015 ACM.

Keywords

  • Approximation theory Convex geometry
  • Kantorovich-Rubinstein duality
  • Mixture learning
  • Randomized algorithms
  • Spectral methods
  • Transportation distance

Fingerprint

Dive into the research topics of 'Learning arbitrary statistical mixtures of discrete distributions'. Together they form a unique fingerprint.

Cite this