Putting peer prediction under the micro(Economic)scope and making truth-telling focal

Yuqing Kong; Katrina Ligett; Grant Schoenebeck

doi:10.1007/978-3-662-54110-4_18

Putting peer prediction under the micro(Economic)scope and making truth-telling focal

Yuqing Kong^*, Katrina Ligett, Grant Schoenebeck

^*Corresponding author for this work

The Rachel and Selim Benin School of Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

21 Scopus citations

Abstract

Peer-prediction [19] is a (meta-)mechanism which, given any proper scoring rule, produces a mechanism to elicit prie information from self-interested agents. Formally, truth-telling is a strict Nash equilibrium of the mechanism. Unfortunately, there may be other equilibria as well (including uninformative equilibria where all players simply report the same fixed signal, regardless of their true signal) and, typically, the truthtelling equilibrium does not have the highest expected payoff. The main result of this paper is to show that, in the symmetric binary setting, by tweaking peer-prediction, in part by carefully selecting the proper scoring rule it is based on, we can make the truth-telling equilibrium focal—that is, truth-telling has higher expected payoff than any other equilibrium. Along the way, we prove the following: in the setting where agents receive binary signals we (1) classify all equilibria of the peer-prediction mechanism; (2) introduce a new technical tool for understanding scoring rules, which allows us to make truth-telling pay better than any other informative equilibrium; (3) leverage this tool to provide an optimal version of the previous result; that is, we optimize the gap between the expected payoff of truth-telling and other informative equilibria; and (4) show that with a slight modification to the peer-prediction framework, we can, in general, make the truth-telling equilibrium focal—that is, truth-telling pays more than any other equilibrium (including the uninformative equilibria).

Original language	American English
Title of host publication	Web and Internet Economics - 12th International Conference, WINE 2016, Proceedings
Editors	Adrian Vetta, Yang Cai
Publisher	Springer Verlag
Pages	251-264
Number of pages	14
ISBN (Print)	9783662541098
DOIs	https://doi.org/10.1007/978-3-662-54110-4_18
State	Published - 2016
Event	12th International Conference on Web and Internet Economics, WINE 2016 - Montreal, Canada Duration: 11 Jun 2016 → 14 Jul 2016

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10123 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	12th International Conference on Web and Internet Economics, WINE 2016
Country/Territory	Canada
City	Montreal
Period	11/06/16 → 14/07/16

Bibliographical note

Funding Information:
K. Ligett—Supported in part NSF grants 1254169 and 1518941, US-Israel Binational Science Foundation Grant 2012348, the Charles Lee Powell Foundation, a subcontract through the Brandeis project, a grant from the HUJI Cyber Security Research Center, and a startup grant from Hebrew Universitys School of Computer Science.

Funding Information:
Y. Kong—Supported by National Science Foundation Career Award 1452915 and CCF Award 1618187.

Funding Information:
G. Schoenebeck—Supported by National Science Foundation Career Award 1452915 and Algorithms in the Field Award 1535912.

Publisher Copyright:
© Springer-Verlag GmbH Germany 2016.

Keywords

Crowdsourcing
Information elicitation
Peer prediction

Access to Document

10.1007/978-3-662-54110-4_18

Cite this

Kong, Y., Ligett, K., & Schoenebeck, G. (2016). Putting peer prediction under the micro(Economic)scope and making truth-telling focal. In A. Vetta, & Y. Cai (Eds.), Web and Internet Economics - 12th International Conference, WINE 2016, Proceedings (pp. 251-264). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10123 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-662-54110-4_18

Kong, Yuqing ; Ligett, Katrina ; Schoenebeck, Grant. / Putting peer prediction under the micro(Economic)scope and making truth-telling focal. Web and Internet Economics - 12th International Conference, WINE 2016, Proceedings. editor / Adrian Vetta ; Yang Cai. Springer Verlag, 2016. pp. 251-264 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{772480a407a8467ba6e4b455470a0364,

title = "Putting peer prediction under the micro(Economic)scope and making truth-telling focal",

abstract = "Peer-prediction [19] is a (meta-)mechanism which, given any proper scoring rule, produces a mechanism to elicit prie information from self-interested agents. Formally, truth-telling is a strict Nash equilibrium of the mechanism. Unfortunately, there may be other equilibria as well (including uninformative equilibria where all players simply report the same fixed signal, regardless of their true signal) and, typically, the truthtelling equilibrium does not have the highest expected payoff. The main result of this paper is to show that, in the symmetric binary setting, by tweaking peer-prediction, in part by carefully selecting the proper scoring rule it is based on, we can make the truth-telling equilibrium focal—that is, truth-telling has higher expected payoff than any other equilibrium. Along the way, we prove the following: in the setting where agents receive binary signals we (1) classify all equilibria of the peer-prediction mechanism; (2) introduce a new technical tool for understanding scoring rules, which allows us to make truth-telling pay better than any other informative equilibrium; (3) leverage this tool to provide an optimal version of the previous result; that is, we optimize the gap between the expected payoff of truth-telling and other informative equilibria; and (4) show that with a slight modification to the peer-prediction framework, we can, in general, make the truth-telling equilibrium focal—that is, truth-telling pays more than any other equilibrium (including the uninformative equilibria).",

keywords = "Crowdsourcing, Information elicitation, Peer prediction",

author = "Yuqing Kong and Katrina Ligett and Grant Schoenebeck",

note = "Funding Information: K. Ligett—Supported in part NSF grants 1254169 and 1518941, US-Israel Binational Science Foundation Grant 2012348, the Charles Lee Powell Foundation, a subcontract through the Brandeis project, a grant from the HUJI Cyber Security Research Center, and a startup grant from Hebrew Universitys School of Computer Science. Funding Information: Y. Kong—Supported by National Science Foundation Career Award 1452915 and CCF Award 1618187. Funding Information: G. Schoenebeck—Supported by National Science Foundation Career Award 1452915 and Algorithms in the Field Award 1535912. Publisher Copyright: {\textcopyright} Springer-Verlag GmbH Germany 2016.; 12th International Conference on Web and Internet Economics, WINE 2016 ; Conference date: 11-06-2016 Through 14-07-2016",

year = "2016",

doi = "10.1007/978-3-662-54110-4_18",

language = "American English",

isbn = "9783662541098",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "251--264",

editor = "Adrian Vetta and Yang Cai",

booktitle = "Web and Internet Economics - 12th International Conference, WINE 2016, Proceedings",

address = "Germany",

}

Kong, Y, Ligett, K & Schoenebeck, G 2016, Putting peer prediction under the micro(Economic)scope and making truth-telling focal. in A Vetta & Y Cai (eds), Web and Internet Economics - 12th International Conference, WINE 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10123 LNCS, Springer Verlag, pp. 251-264, 12th International Conference on Web and Internet Economics, WINE 2016, Montreal, Canada, 11/06/16. https://doi.org/10.1007/978-3-662-54110-4_18

Putting peer prediction under the micro(Economic)scope and making truth-telling focal. / Kong, Yuqing; Ligett, Katrina; Schoenebeck, Grant.
Web and Internet Economics - 12th International Conference, WINE 2016, Proceedings. ed. / Adrian Vetta; Yang Cai. Springer Verlag, 2016. p. 251-264 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10123 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Putting peer prediction under the micro(Economic)scope and making truth-telling focal

AU - Kong, Yuqing

AU - Ligett, Katrina

AU - Schoenebeck, Grant

N1 - Funding Information: K. Ligett—Supported in part NSF grants 1254169 and 1518941, US-Israel Binational Science Foundation Grant 2012348, the Charles Lee Powell Foundation, a subcontract through the Brandeis project, a grant from the HUJI Cyber Security Research Center, and a startup grant from Hebrew Universitys School of Computer Science. Funding Information: Y. Kong—Supported by National Science Foundation Career Award 1452915 and CCF Award 1618187. Funding Information: G. Schoenebeck—Supported by National Science Foundation Career Award 1452915 and Algorithms in the Field Award 1535912. Publisher Copyright: © Springer-Verlag GmbH Germany 2016.

PY - 2016

Y1 - 2016

N2 - Peer-prediction [19] is a (meta-)mechanism which, given any proper scoring rule, produces a mechanism to elicit prie information from self-interested agents. Formally, truth-telling is a strict Nash equilibrium of the mechanism. Unfortunately, there may be other equilibria as well (including uninformative equilibria where all players simply report the same fixed signal, regardless of their true signal) and, typically, the truthtelling equilibrium does not have the highest expected payoff. The main result of this paper is to show that, in the symmetric binary setting, by tweaking peer-prediction, in part by carefully selecting the proper scoring rule it is based on, we can make the truth-telling equilibrium focal—that is, truth-telling has higher expected payoff than any other equilibrium. Along the way, we prove the following: in the setting where agents receive binary signals we (1) classify all equilibria of the peer-prediction mechanism; (2) introduce a new technical tool for understanding scoring rules, which allows us to make truth-telling pay better than any other informative equilibrium; (3) leverage this tool to provide an optimal version of the previous result; that is, we optimize the gap between the expected payoff of truth-telling and other informative equilibria; and (4) show that with a slight modification to the peer-prediction framework, we can, in general, make the truth-telling equilibrium focal—that is, truth-telling pays more than any other equilibrium (including the uninformative equilibria).

AB - Peer-prediction [19] is a (meta-)mechanism which, given any proper scoring rule, produces a mechanism to elicit prie information from self-interested agents. Formally, truth-telling is a strict Nash equilibrium of the mechanism. Unfortunately, there may be other equilibria as well (including uninformative equilibria where all players simply report the same fixed signal, regardless of their true signal) and, typically, the truthtelling equilibrium does not have the highest expected payoff. The main result of this paper is to show that, in the symmetric binary setting, by tweaking peer-prediction, in part by carefully selecting the proper scoring rule it is based on, we can make the truth-telling equilibrium focal—that is, truth-telling has higher expected payoff than any other equilibrium. Along the way, we prove the following: in the setting where agents receive binary signals we (1) classify all equilibria of the peer-prediction mechanism; (2) introduce a new technical tool for understanding scoring rules, which allows us to make truth-telling pay better than any other informative equilibrium; (3) leverage this tool to provide an optimal version of the previous result; that is, we optimize the gap between the expected payoff of truth-telling and other informative equilibria; and (4) show that with a slight modification to the peer-prediction framework, we can, in general, make the truth-telling equilibrium focal—that is, truth-telling pays more than any other equilibrium (including the uninformative equilibria).

KW - Crowdsourcing

KW - Information elicitation

KW - Peer prediction

UR - http://www.scopus.com/inward/record.url?scp=85007300058&partnerID=8YFLogxK

U2 - 10.1007/978-3-662-54110-4_18

DO - 10.1007/978-3-662-54110-4_18

M3 - Conference contribution

AN - SCOPUS:85007300058

SN - 9783662541098

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 251

EP - 264

BT - Web and Internet Economics - 12th International Conference, WINE 2016, Proceedings

A2 - Vetta, Adrian

A2 - Cai, Yang

PB - Springer Verlag

T2 - 12th International Conference on Web and Internet Economics, WINE 2016

Y2 - 11 June 2016 through 14 July 2016

ER -

Kong Y, Ligett K, Schoenebeck G. Putting peer prediction under the micro(Economic)scope and making truth-telling focal. In Vetta A, Cai Y, editors, Web and Internet Economics - 12th International Conference, WINE 2016, Proceedings. Springer Verlag. 2016. p. 251-264. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-662-54110-4_18

Putting peer prediction under the micro(Economic)scope and making truth-telling focal

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this