Online Classification for Complex Problems Using Simultaneous Projections

Yonatan Amit, Shai Shalev-Shwartz, Yoram Singer

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

We describe and analyze an algorithmic framework for online classification where each online trial consists of multiple prediction tasks that are tied together. We tackle the problem of updating the online hypothesis by defining a projection problem in which each prediction task corresponds to a single linear constraint. These constraints are tied together through a single slack parameter. We then introduce a general method for approximately solving the problem by projecting simultaneously and independently on each constraint which corresponds to a prediction sub-problem, and then averaging the individual solutions. We show that this approach constitutes a feasible, albeit not necessarily optimal, solution for the original projection problem. We derive concrete simultaneous projection schemes and analyze them in the mistake bound model. We demonstrate the power of the proposed algorithm in experiments with online multiclass text categorization. Our experiments indicate that a combination of class-dependent features with the simultaneous projection method outperforms previously studied algorithms.

Original languageEnglish
Title of host publicationNIPS 2006
Subtitle of host publicationProceedings of the 19th International Conference on Neural Information Processing Systems
EditorsBernhard Scholkopf, John C. Platt, Thomas Hofmann
PublisherMIT Press Journals
Pages17-24
Number of pages8
ISBN (Electronic)0262195682, 9780262195683
StatePublished - 2006
Event19th International Conference on Neural Information Processing Systems, NIPS 2006 - Vancouver, Canada
Duration: 4 Dec 20067 Dec 2006

Publication series

NameNIPS 2006: Proceedings of the 19th International Conference on Neural Information Processing Systems

Conference

Conference19th International Conference on Neural Information Processing Systems, NIPS 2006
Country/TerritoryCanada
CityVancouver
Period4/12/067/12/06

Bibliographical note

Publisher Copyright:
© NIPS 2006.All rights reserved

Fingerprint

Dive into the research topics of 'Online Classification for Complex Problems Using Simultaneous Projections'. Together they form a unique fingerprint.

Cite this