Efficient bandit algorithms for online multiclass prediction

Sham M. Kakade, Shai Shalev-Shwartz, Ambuj Tewari

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

95 Scopus citations

Abstract

This paper introduces the Banditron, a variant of the Perception [Rosenblatt, 1958], for the multiclass bandit setting. The multiclass bandit setting models a wide range of practical supervised learning applications where the learner only receives partial feedback (referred to as "bandit" feedback, in the spirit of multi-armed bandit models) with respect to the true label (e.g. in many web applications users often only provide positive "click" feedback which does not necessarily fully disclose a true label). The Banditron has the ability to learn in a multiclass classification setting with the "bandit" feedback which only reveals whether or not the prediction made by the algorithm was correct or not (but does not necessarily reveal the true label). We pro vide (relative) mistake bounds which show how the Banditron enjoys favorable performance, and our experiments demonstrate the practicality of the algorithm. Furthermore, this paper pays close attention to the important special case when the data is linearly separable - a problem which has been exhaustively studied in the full information setting yet is novel in the bandit setting.

Original languageAmerican English
Title of host publicationProceedings of the 25th International Conference on Machine Learning
PublisherAssociation for Computing Machinery (ACM)
Pages440-447
Number of pages8
ISBN (Print)9781605582054
DOIs
StatePublished - 2008
Externally publishedYes
Event25th International Conference on Machine Learning - Helsinki, Finland
Duration: 5 Jul 20089 Jul 2008

Publication series

NameProceedings of the 25th International Conference on Machine Learning

Conference

Conference25th International Conference on Machine Learning
Country/TerritoryFinland
CityHelsinki
Period5/07/089/07/08

Fingerprint

Dive into the research topics of 'Efficient bandit algorithms for online multiclass prediction'. Together they form a unique fingerprint.

Cite this