Knowledge Discovery in Databases (KDD) focuses on the computerized exploration of large amounts of data and on the discovery of interesting patterns within them. While most work on KDD has been concerned with structured databases, there has been little work on handling the huge amount of information that is available only in unstructured document collections. This paper describes a new method for computing co-occurrence frequencies of the various keywords labeling the documents. This method is based on computing maximal association rules. Regular associations are based on the notion of frequent sets: sets of attributes, which appear in many records. In analogy, maximal associations are based on the notion of frequent maximal sets. Conceptually, a frequent maximal set is a set of attributes, which appear alone, or maximally, in many records. For the definition of "maximality"we use an underlying taxonomy, T, of the attributes. This allows us to obtain the "interesting"correlations between attributes from different categories. Frequent maximal sets are useful for efficiently finding association rules that include negated attributes. We provide an experimental evaluation of our methodology on the Reuters-21578 document collection.
|Original language||American English|
|Title of host publication||Proceedings - 3rd International Conference on Knowledge Discovery and Data Mining, KDD 1997|
|Editors||David Heckerman, Heikki Mannila, Daryl Pregibon, Ramasamy Uthurusamy|
|Number of pages||4|
|ISBN (Electronic)||1577350278, 9781577350279|
|State||Published - 1997|
|Event||3rd International Conference on Knowledge Discovery and Data Mining, KDD 1997 - Newport Beach, United States|
Duration: 14 Aug 1997 → 17 Aug 1997
|Name||Proceedings - 3rd International Conference on Knowledge Discovery and Data Mining, KDD 1997|
|Conference||3rd International Conference on Knowledge Discovery and Data Mining, KDD 1997|
|Period||14/08/97 → 17/08/97|
Bibliographical noteFunding Information:
This research was supported by grant # 8615 from the Israeli Ministry of Sciences, and by a GIF grant 1-429-096.
Copyright © 1997, American Association for Artificial Intelligence (www.aaai.org). All rights reserved.