Learning a Mahalanobis metric from equivalence constraints

Aharon Bar-Hillel*, Tomer Hertz, Noam Shental, Daphna Weinshall

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

476 Scopus citations

Abstract

Many learning algorithms use a metric defined over the input space as a principal tool, and their performance critically depends on the quality of this metric. We address the problem of learning metrics using side-information in the form of equivalence constraints. Unlike labels, we demonstrate that this type of side-information can sometimes be automatically obtained without the need of human intervention. We show how such side-information can be used to modify the representation of the data, leading to improved clustering and classification. Specifically, we present the Relevant Component Analysis (RCA) algorithm, which is a simple and efficient algorithm for learning a Mahalanobis metric. We show that RCA is the solution of an interesting optimization problem, founded on an information theoretic basis. If dimensionality reduction is allowed within RCA, we show that it is optimally accomplished by a version of Fisher's linear discriminant that uses constraints. Moreover, under certain Gaussian assumptions, RCA can be viewed as a Maximum Likelihood estimation of the within class covariance matrix. We conclude with extensive empirical evaluations of RCA, showing its advantage over alternative methods.

Original languageAmerican English
JournalJournal of Machine Learning Research
Volume6
StatePublished - 2005

Keywords

  • Clustering
  • Dimensionality reduction
  • Equivalence constraints
  • Metric learning
  • Side information

Fingerprint

Dive into the research topics of 'Learning a Mahalanobis metric from equivalence constraints'. Together they form a unique fingerprint.

Cite this