The future of open human feedback

Shachar Don-Yehiya; Ben Burtenshaw; Ramon Fernandez Astudillo; Cailean Osborne; Mimansa Jaiswal; Tzu Sheng Kuo; Wenting Zhao; Idan Shenfeld; Andi Peng; Mikhail Yurochkin; Atoosa Kasirzadeh; Yangsibo Huang; Tatsunori Hashimoto; Yacine Jernite; Daniel Vila-Suero; Omri Abend; Jennifer Ding; Sara Hooker; Hannah Rose Kirk; Leshem Choshen

doi:10.1038/s42256-025-01038-2

The future of open human feedback

Shachar Don-Yehiya, Ben Burtenshaw, Ramon Fernandez Astudillo, Cailean Osborne, Mimansa Jaiswal, Tzu Sheng Kuo, Wenting Zhao, Idan Shenfeld, Andi Peng, Mikhail Yurochkin, Atoosa Kasirzadeh, Yangsibo Huang, Tatsunori Hashimoto, Yacine Jernite, Daniel Vila-Suero, Omri Abend, Jennifer Ding, Sara Hooker, Hannah Rose Kirk^*, Leshem Choshen^*

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

Abstract

Human feedback on conversations with language models is central to how these systems learn about the world, improve their capabilities and are steered towards desirable and safe behaviours. However, this feedback is mostly collected by frontier artificial intelligence labs and kept behind closed doors. Here we bring together interdisciplinary experts to assess the opportunities and challenges to realizing an open ecosystem of human feedback for artificial intelligence. We first look for successful practices in the peer-production, open-source and citizen-science communities. We then characterize the main challenges for open human feedback. For each, we survey current approaches and offer recommendations. We end by envisioning the components needed to underpin a sustainable and open human feedback ecosystem. In the centre of this ecosystem are mutually beneficial feedback loops, between users and specialized models, incentivizing a diverse stakeholder community of model trainers and feedback providers to support a general open feedback pool.

Original language	English
Pages (from-to)	825-835
Number of pages	11
Journal	Nature Machine Intelligence
Volume	7
Issue number	6
DOIs	https://doi.org/10.1038/s42256-025-01038-2
State	Published - Jun 2025

Bibliographical note

Publisher Copyright:
© Springer Nature Limited 2025.

Access to Document

10.1038/s42256-025-01038-2

Cite this

Don-Yehiya, S., Burtenshaw, B., Fernandez Astudillo, R., Osborne, C., Jaiswal, M., Kuo, T. S., Zhao, W., Shenfeld, I., Peng, A., Yurochkin, M., Kasirzadeh, A., Huang, Y., Hashimoto, T., Jernite, Y., Vila-Suero, D., Abend, O., Ding, J., Hooker, S., Rose Kirk, H., & Choshen, L. (2025). The future of open human feedback. Nature Machine Intelligence, 7(6), 825-835. https://doi.org/10.1038/s42256-025-01038-2

@article{0312fa8202b74426bcd12649449606c6,

title = "The future of open human feedback",

abstract = "Human feedback on conversations with language models is central to how these systems learn about the world, improve their capabilities and are steered towards desirable and safe behaviours. However, this feedback is mostly collected by frontier artificial intelligence labs and kept behind closed doors. Here we bring together interdisciplinary experts to assess the opportunities and challenges to realizing an open ecosystem of human feedback for artificial intelligence. We first look for successful practices in the peer-production, open-source and citizen-science communities. We then characterize the main challenges for open human feedback. For each, we survey current approaches and offer recommendations. We end by envisioning the components needed to underpin a sustainable and open human feedback ecosystem. In the centre of this ecosystem are mutually beneficial feedback loops, between users and specialized models, incentivizing a diverse stakeholder community of model trainers and feedback providers to support a general open feedback pool.",

author = "Shachar Don-Yehiya and Ben Burtenshaw and \{Fernandez Astudillo\}, Ramon and Cailean Osborne and Mimansa Jaiswal and Kuo, \{Tzu Sheng\} and Wenting Zhao and Idan Shenfeld and Andi Peng and Mikhail Yurochkin and Atoosa Kasirzadeh and Yangsibo Huang and Tatsunori Hashimoto and Yacine Jernite and Daniel Vila-Suero and Omri Abend and Jennifer Ding and Sara Hooker and \{Rose Kirk\}, Hannah and Leshem Choshen",

note = "Publisher Copyright: {\textcopyright} Springer Nature Limited 2025.",

year = "2025",

month = jun,

doi = "10.1038/s42256-025-01038-2",

language = "אנגלית",

volume = "7",

pages = "825--835",

journal = "Nature Machine Intelligence",

issn = "2522-5839",

publisher = "Springer International Publishing",

number = "6",

}

Don-Yehiya, S, Burtenshaw, B, Fernandez Astudillo, R, Osborne, C, Jaiswal, M, Kuo, TS, Zhao, W, Shenfeld, I, Peng, A, Yurochkin, M, Kasirzadeh, A, Huang, Y, Hashimoto, T, Jernite, Y, Vila-Suero, D, Abend, O, Ding, J, Hooker, S, Rose Kirk, H & Choshen, L 2025, 'The future of open human feedback', Nature Machine Intelligence, vol. 7, no. 6, pp. 825-835. https://doi.org/10.1038/s42256-025-01038-2

TY - JOUR

T1 - The future of open human feedback

AU - Don-Yehiya, Shachar

AU - Burtenshaw, Ben

AU - Fernandez Astudillo, Ramon

AU - Osborne, Cailean

AU - Jaiswal, Mimansa

AU - Kuo, Tzu Sheng

AU - Zhao, Wenting

AU - Shenfeld, Idan

AU - Peng, Andi

AU - Yurochkin, Mikhail

AU - Kasirzadeh, Atoosa

AU - Huang, Yangsibo

AU - Hashimoto, Tatsunori

AU - Jernite, Yacine

AU - Vila-Suero, Daniel

AU - Abend, Omri

AU - Ding, Jennifer

AU - Hooker, Sara

AU - Rose Kirk, Hannah

AU - Choshen, Leshem

PY - 2025/6

Y1 - 2025/6

N2 - Human feedback on conversations with language models is central to how these systems learn about the world, improve their capabilities and are steered towards desirable and safe behaviours. However, this feedback is mostly collected by frontier artificial intelligence labs and kept behind closed doors. Here we bring together interdisciplinary experts to assess the opportunities and challenges to realizing an open ecosystem of human feedback for artificial intelligence. We first look for successful practices in the peer-production, open-source and citizen-science communities. We then characterize the main challenges for open human feedback. For each, we survey current approaches and offer recommendations. We end by envisioning the components needed to underpin a sustainable and open human feedback ecosystem. In the centre of this ecosystem are mutually beneficial feedback loops, between users and specialized models, incentivizing a diverse stakeholder community of model trainers and feedback providers to support a general open feedback pool.

AB - Human feedback on conversations with language models is central to how these systems learn about the world, improve their capabilities and are steered towards desirable and safe behaviours. However, this feedback is mostly collected by frontier artificial intelligence labs and kept behind closed doors. Here we bring together interdisciplinary experts to assess the opportunities and challenges to realizing an open ecosystem of human feedback for artificial intelligence. We first look for successful practices in the peer-production, open-source and citizen-science communities. We then characterize the main challenges for open human feedback. For each, we survey current approaches and offer recommendations. We end by envisioning the components needed to underpin a sustainable and open human feedback ecosystem. In the centre of this ecosystem are mutually beneficial feedback loops, between users and specialized models, incentivizing a diverse stakeholder community of model trainers and feedback providers to support a general open feedback pool.

UR - http://www.scopus.com/inward/record.url?scp=105008684248&partnerID=8YFLogxK

U2 - 10.1038/s42256-025-01038-2

DO - 10.1038/s42256-025-01038-2

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???

AN - SCOPUS:105008684248

SN - 2522-5839

VL - 7

SP - 825

EP - 835

JO - Nature Machine Intelligence

JF - Nature Machine Intelligence

IS - 6

ER -

The future of open human feedback

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this