TY - GEN
T1 - Mind the duality gap
T2 - 22nd Annual Conference on Neural Information Processing Systems, NIPS 2008
AU - Kakade, Sham M.
AU - Shalev-Shwartz, Shai
PY - 2009
Y1 - 2009
N2 - We describe a primal-dual framework for the design and analysis of online strongly convex optimization algorithms. Our framework yields the tightest known logarithmic regret bounds for Follow-The-Leader and for the gradient descent algorithm proposed in Hazan et al. [2006]. We then show that one can interpolate between these two extreme cases. In particular, we derive a new algorithm that shares the computational simplicity of gradient descent but achieves lower regret in many practical situations. Finally, we further extend our framework for generalized strongly convex functions.
AB - We describe a primal-dual framework for the design and analysis of online strongly convex optimization algorithms. Our framework yields the tightest known logarithmic regret bounds for Follow-The-Leader and for the gradient descent algorithm proposed in Hazan et al. [2006]. We then show that one can interpolate between these two extreme cases. In particular, we derive a new algorithm that shares the computational simplicity of gradient descent but achieves lower regret in many practical situations. Finally, we further extend our framework for generalized strongly convex functions.
UR - http://www.scopus.com/inward/record.url?scp=77951165785&partnerID=8YFLogxK
M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???
AN - SCOPUS:77951165785
SN - 9781605609492
T3 - Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference
SP - 1457
EP - 1464
BT - Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference
PB - Neural Information Processing Systems
Y2 - 8 December 2008 through 11 December 2008
ER -