Optimal histograms with outliers

Rachel Behar; Sara Cohen

doi:10.5441/002/edbt.2020.17

Optimal histograms with outliers

Rachel Behar, Sara Cohen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Histograms are a well studied and simple way to summarize data. As such, they are used extensively in a variety of applications that require estimates of data frequency values. Significant previous work has studied the problem of finding optimal histograms with respect to an error measure. In this paper we study the classic problem of finding an optimal histogram for a dataset, with a new twist: The histogram must contain at least n − k of the n data points. The k excluded data points are considered outliers. We consider two notions of excluding data items, by allowing arbitrary items to be excluded, or only removing items while retaining a consistent histogram. Polynomial algorithms are presented for these problems. Significant experimentation demonstrates that our algorithms work well in practice to reduce the histogram error.

Original language	English
Title of host publication	Advances in Database Technology - EDBT 2020
Subtitle of host publication	23rd International Conference on Extending Database Technology, Proceedings
Editors	Angela Bonifati, Yongluan Zhou, Marcos Antonio Vaz Salles, Alexander Bohm, Dan Olteanu, George Fletcher, Arijit Khan, Bin Yang
Publisher	OpenProceedings.org
Pages	181-192
Number of pages	12
ISBN (Electronic)	9783893180837
DOIs	https://doi.org/10.5441/002/edbt.2020.17
State	Published - 2020
Externally published	Yes
Event	23rd International Conference on Extending Database Technology, EDBT 2020 - Copenhagen, Denmark Duration: 30 Mar 2020 → 2 Apr 2020

Publication series

Name	Advances in Database Technology - EDBT
Volume	2020-March
ISSN (Electronic)	2367-2005

Conference

Conference	23rd International Conference on Extending Database Technology, EDBT 2020
Country/Territory	Denmark
City	Copenhagen
Period	30/03/20 → 2/04/20

Bibliographical note

Publisher Copyright:
© 2020 Copyright held by the owner/author(s).

Access to Document

10.5441/002/edbt.2020.17

Cite this

Behar, R., & Cohen, S. (2020). Optimal histograms with outliers. In A. Bonifati, Y. Zhou, M. A. Vaz Salles, A. Bohm, D. Olteanu, G. Fletcher, A. Khan, & B. Yang (Eds.), Advances in Database Technology - EDBT 2020: 23rd International Conference on Extending Database Technology, Proceedings (pp. 181-192). (Advances in Database Technology - EDBT; Vol. 2020-March). OpenProceedings.org. https://doi.org/10.5441/002/edbt.2020.17

Behar, Rachel ; Cohen, Sara. / Optimal histograms with outliers. Advances in Database Technology - EDBT 2020: 23rd International Conference on Extending Database Technology, Proceedings. editor / Angela Bonifati ; Yongluan Zhou ; Marcos Antonio Vaz Salles ; Alexander Bohm ; Dan Olteanu ; George Fletcher ; Arijit Khan ; Bin Yang. OpenProceedings.org, 2020. pp. 181-192 (Advances in Database Technology - EDBT).

@inproceedings{6ba2d06a23d2426aa3cb225a988342b5,

title = "Optimal histograms with outliers",

abstract = "Histograms are a well studied and simple way to summarize data. As such, they are used extensively in a variety of applications that require estimates of data frequency values. Significant previous work has studied the problem of finding optimal histograms with respect to an error measure. In this paper we study the classic problem of finding an optimal histogram for a dataset, with a new twist: The histogram must contain at least n − k of the n data points. The k excluded data points are considered outliers. We consider two notions of excluding data items, by allowing arbitrary items to be excluded, or only removing items while retaining a consistent histogram. Polynomial algorithms are presented for these problems. Significant experimentation demonstrates that our algorithms work well in practice to reduce the histogram error.",

author = "Rachel Behar and Sara Cohen",

note = "Publisher Copyright: {\textcopyright} 2020 Copyright held by the owner/author(s).; 23rd International Conference on Extending Database Technology, EDBT 2020 ; Conference date: 30-03-2020 Through 02-04-2020",

year = "2020",

doi = "10.5441/002/edbt.2020.17",

language = "אנגלית",

series = "Advances in Database Technology - EDBT",

publisher = "OpenProceedings.org",

pages = "181--192",

editor = "Angela Bonifati and Yongluan Zhou and {Vaz Salles}, {Marcos Antonio} and Alexander Bohm and Dan Olteanu and George Fletcher and Arijit Khan and Bin Yang",

booktitle = "Advances in Database Technology - EDBT 2020",

}

Behar, R & Cohen, S 2020, Optimal histograms with outliers. in A Bonifati, Y Zhou, MA Vaz Salles, A Bohm, D Olteanu, G Fletcher, A Khan & B Yang (eds), Advances in Database Technology - EDBT 2020: 23rd International Conference on Extending Database Technology, Proceedings. Advances in Database Technology - EDBT, vol. 2020-March, OpenProceedings.org, pp. 181-192, 23rd International Conference on Extending Database Technology, EDBT 2020, Copenhagen, Denmark, 30/03/20. https://doi.org/10.5441/002/edbt.2020.17

Optimal histograms with outliers. / Behar, Rachel; Cohen, Sara.
Advances in Database Technology - EDBT 2020: 23rd International Conference on Extending Database Technology, Proceedings. ed. / Angela Bonifati; Yongluan Zhou; Marcos Antonio Vaz Salles; Alexander Bohm; Dan Olteanu; George Fletcher; Arijit Khan; Bin Yang. OpenProceedings.org, 2020. p. 181-192 (Advances in Database Technology - EDBT; Vol. 2020-March).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Optimal histograms with outliers

AU - Behar, Rachel

AU - Cohen, Sara

PY - 2020

Y1 - 2020

N2 - Histograms are a well studied and simple way to summarize data. As such, they are used extensively in a variety of applications that require estimates of data frequency values. Significant previous work has studied the problem of finding optimal histograms with respect to an error measure. In this paper we study the classic problem of finding an optimal histogram for a dataset, with a new twist: The histogram must contain at least n − k of the n data points. The k excluded data points are considered outliers. We consider two notions of excluding data items, by allowing arbitrary items to be excluded, or only removing items while retaining a consistent histogram. Polynomial algorithms are presented for these problems. Significant experimentation demonstrates that our algorithms work well in practice to reduce the histogram error.

AB - Histograms are a well studied and simple way to summarize data. As such, they are used extensively in a variety of applications that require estimates of data frequency values. Significant previous work has studied the problem of finding optimal histograms with respect to an error measure. In this paper we study the classic problem of finding an optimal histogram for a dataset, with a new twist: The histogram must contain at least n − k of the n data points. The k excluded data points are considered outliers. We consider two notions of excluding data items, by allowing arbitrary items to be excluded, or only removing items while retaining a consistent histogram. Polynomial algorithms are presented for these problems. Significant experimentation demonstrates that our algorithms work well in practice to reduce the histogram error.

UR - http://www.scopus.com/inward/record.url?scp=85084181165&partnerID=8YFLogxK

U2 - 10.5441/002/edbt.2020.17

DO - 10.5441/002/edbt.2020.17

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85084181165

T3 - Advances in Database Technology - EDBT

SP - 181

EP - 192

BT - Advances in Database Technology - EDBT 2020

A2 - Bonifati, Angela

A2 - Zhou, Yongluan

A2 - Vaz Salles, Marcos Antonio

A2 - Bohm, Alexander

A2 - Olteanu, Dan

A2 - Fletcher, George

A2 - Khan, Arijit

A2 - Yang, Bin

PB - OpenProceedings.org

T2 - 23rd International Conference on Extending Database Technology, EDBT 2020

Y2 - 30 March 2020 through 2 April 2020

ER -

Behar R, Cohen S. Optimal histograms with outliers. In Bonifati A, Zhou Y, Vaz Salles MA, Bohm A, Olteanu D, Fletcher G, Khan A, Yang B, editors, Advances in Database Technology - EDBT 2020: 23rd International Conference on Extending Database Technology, Proceedings. OpenProceedings.org. 2020. p. 181-192. (Advances in Database Technology - EDBT). doi: 10.5441/002/edbt.2020.17

Optimal histograms with outliers

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this