ID3 Learns Juntas for Smoothed Product Distributions

Alon Brtuzkus, Amit Daniely, Eran Malach

Research output: Contribution to journalConference articlepeer-review

12 Scopus citations

Abstract

In recent years, there are many attempts to understand popular heuristics. An example of such heuristic algorithm is the ID3 algorithm for learning decision trees. This algorithm is commonly used in practice, but there are very few theoretical works studying its behavior. In this paper, we analyze the ID3 algorithm, when the target function is a k-Junta, a function that depends on k out of n variables of the input. We prove that when k = log n, the ID3 algorithm learns in polynomial time k-Juntas, in the smoothed analysis model of Kalai and Teng (2008). That is, we show a learnability result when the observed distribution is a “noisy” variant of the original distribution.

Original languageAmerican English
Pages (from-to)902-915
Number of pages14
JournalProceedings of Machine Learning Research
Volume125
StatePublished - 2020
Event33rd Conference on Learning Theory, COLT 2020 - Virtual, Online, Austria
Duration: 9 Jul 202012 Jul 2020

Bibliographical note

Funding Information:
This research was supported by the Yandex Initiative in Machine Learning.

Publisher Copyright:
© 2020 A. Brtuzkus, A. Daniely & E. Malach.

Keywords

  • Decision Trees
  • ID3
  • Junta
  • Parity

Fingerprint

Dive into the research topics of 'ID3 Learns Juntas for Smoothed Product Distributions'. Together they form a unique fingerprint.

Cite this