Learning halfspaces with the zero-one loss: Time-accuracy tradeoffs

Given α, ε, we study the time complexity required to improperly learn a halfs-pace with misclassification error rate of at most (1 + α)L_*^gamma + ε, where L _*^gamma is the optimal γ-margin error rate. For α = 1/γ, polynomial time and sample complexity is achievable using the hinge-loss. For α = 0, Shalev-Shwartz et al. [2011] showed that poly(1/γ) time is impossible, while learning is possible in time exp(Õ (1/γ)). An immediate question, which this paper tackles, is what is achievable if α ε (0, 1/γ). We derive positive results interpolating between the polynomial time for α = 1/γ and the exponential time for α = 0. In particular, we show that there are cases in which α = o(1/γ) but the problem is still solvable in polynomial time. Our results naturally extend to the adversarial online learning model and to the PAC learning with malicious noise model.