Refining Labeling Functions with Limited Labeled Data

Chenjie Li; Amir Gilad; Boris Glavic; Zhengjie Miao; Sudeepa Roy

doi:10.1145/3711896.3737102

Refining Labeling Functions with Limited Labeled Data

Chenjie Li
, Amir Gilad
, Boris Glavic
, Zhengjie Miao
, Sudeepa Roy

The Rachel and Selim Benin School of Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Programmatic weak supervision (PWS) significantly reduces human effort for labeling data by combining the outputs of user-provided labeling functions (LFs) on unlabeled datapoints. However, the quality of the generated labels depends directly on the accuracy of the LFs. In this work, we study the problem of fixing LFs based on a small set of labeled examples. Towards this goal, we develop novel techniques for repairing a set of LFs by minimally changing their results on the labeled examples such that the fixed LFs ensure that (i) there is sufficient evidence for the correct label of each labeled datapoint and (ii) the accuracy of each repaired LF is sufficiently high. We model LFs as conditional rules, which enables us to refine them, i.e., to selectively change their output for some inputs. We demonstrate experimentally that our system improves the quality of LFs based on surprisingly small sets of labeled datapoints.

Original language	English
Title of host publication	KDD 2025 - Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining
Publisher	Association for Computing Machinery
Pages	1318-1329
Number of pages	12
ISBN (Electronic)	9798400714542
DOIs	https://doi.org/10.1145/3711896.3737102
State	Published - 3 Aug 2025
Event	31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2025 - Toronto, Canada Duration: 3 Aug 2025 → 7 Aug 2025

Publication series

Name	Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
Volume	2
ISSN (Print)	2154-817X

Conference

Conference	31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2025
Country/Territory	Canada
City	Toronto
Period	3/08/25 → 7/08/25

Bibliographical note

Publisher Copyright:
© 2025 Copyright held by the owner/author(s). Publication rights licensed to ACM.

Keywords

label quality
label repair
labeling functions
rule refinement
weak supervision

Access to Document

10.1145/3711896.3737102

Cite this

Li, C., Gilad, A., Glavic, B., Miao, Z., & Roy, S. (2025). Refining Labeling Functions with Limited Labeled Data. In KDD 2025 - Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 1318-1329). (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; Vol. 2). Association for Computing Machinery. https://doi.org/10.1145/3711896.3737102

@inproceedings{d504b80686d041d8a0e467087f7fd815,

title = "Refining Labeling Functions with Limited Labeled Data",

abstract = "Programmatic weak supervision (PWS) significantly reduces human effort for labeling data by combining the outputs of user-provided labeling functions (LFs) on unlabeled datapoints. However, the quality of the generated labels depends directly on the accuracy of the LFs. In this work, we study the problem of fixing LFs based on a small set of labeled examples. Towards this goal, we develop novel techniques for repairing a set of LFs by minimally changing their results on the labeled examples such that the fixed LFs ensure that (i) there is sufficient evidence for the correct label of each labeled datapoint and (ii) the accuracy of each repaired LF is sufficiently high. We model LFs as conditional rules, which enables us to refine them, i.e., to selectively change their output for some inputs. We demonstrate experimentally that our system improves the quality of LFs based on surprisingly small sets of labeled datapoints.",

keywords = "label quality, label repair, labeling functions, rule refinement, weak supervision",

author = "Chenjie Li and Amir Gilad and Boris Glavic and Zhengjie Miao and Sudeepa Roy",

note = "Publisher Copyright: {\textcopyright} 2025 Copyright held by the owner/author(s). Publication rights licensed to ACM.; 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2025 ; Conference date: 03-08-2025 Through 07-08-2025",

year = "2025",

month = aug,

day = "3",

doi = "10.1145/3711896.3737102",

language = "אנגלית",

series = "Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining",

publisher = "Association for Computing Machinery",

pages = "1318--1329",

booktitle = "KDD 2025 - Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining",

}

Li, C, Gilad, A, Glavic, B, Miao, Z & Roy, S 2025, Refining Labeling Functions with Limited Labeled Data. in KDD 2025 - Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, vol. 2, Association for Computing Machinery, pp. 1318-1329, 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2025, Toronto, Canada, 3/08/25. https://doi.org/10.1145/3711896.3737102

Refining Labeling Functions with Limited Labeled Data. / Li, Chenjie; Gilad, Amir; Glavic, Boris et al.
KDD 2025 - Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, 2025. p. 1318-1329 (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; Vol. 2).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Refining Labeling Functions with Limited Labeled Data

AU - Li, Chenjie

AU - Gilad, Amir

AU - Glavic, Boris

AU - Miao, Zhengjie

AU - Roy, Sudeepa

PY - 2025/8/3

Y1 - 2025/8/3

N2 - Programmatic weak supervision (PWS) significantly reduces human effort for labeling data by combining the outputs of user-provided labeling functions (LFs) on unlabeled datapoints. However, the quality of the generated labels depends directly on the accuracy of the LFs. In this work, we study the problem of fixing LFs based on a small set of labeled examples. Towards this goal, we develop novel techniques for repairing a set of LFs by minimally changing their results on the labeled examples such that the fixed LFs ensure that (i) there is sufficient evidence for the correct label of each labeled datapoint and (ii) the accuracy of each repaired LF is sufficiently high. We model LFs as conditional rules, which enables us to refine them, i.e., to selectively change their output for some inputs. We demonstrate experimentally that our system improves the quality of LFs based on surprisingly small sets of labeled datapoints.

AB - Programmatic weak supervision (PWS) significantly reduces human effort for labeling data by combining the outputs of user-provided labeling functions (LFs) on unlabeled datapoints. However, the quality of the generated labels depends directly on the accuracy of the LFs. In this work, we study the problem of fixing LFs based on a small set of labeled examples. Towards this goal, we develop novel techniques for repairing a set of LFs by minimally changing their results on the labeled examples such that the fixed LFs ensure that (i) there is sufficient evidence for the correct label of each labeled datapoint and (ii) the accuracy of each repaired LF is sufficiently high. We model LFs as conditional rules, which enables us to refine them, i.e., to selectively change their output for some inputs. We demonstrate experimentally that our system improves the quality of LFs based on surprisingly small sets of labeled datapoints.

KW - label quality

KW - label repair

KW - labeling functions

KW - rule refinement

KW - weak supervision

UR - https://www.scopus.com/pages/publications/105014313332

U2 - 10.1145/3711896.3737102

DO - 10.1145/3711896.3737102

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:105014313332

T3 - Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

SP - 1318

EP - 1329

BT - KDD 2025 - Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

PB - Association for Computing Machinery

T2 - 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2025

Y2 - 3 August 2025 through 7 August 2025

ER -

Li C, Gilad A, Glavic B, Miao Z, Roy S. Refining Labeling Functions with Limited Labeled Data. In KDD 2025 - Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery. 2025. p. 1318-1329. (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining). doi: 10.1145/3711896.3737102

Refining Labeling Functions with Limited Labeled Data

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this