Considerations and Pitfalls in Controlled Experiments on Code Comprehension

Dror G. Feitelson

doi:10.1109/ICPC52881.2021.00019

Considerations and Pitfalls in Controlled Experiments on Code Comprehension

Dror G. Feitelson^*

^*Corresponding author for this work

The Rachel and Selim Benin School of Engineering and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

15 Scopus citations

Abstract

Understanding program code is a complicated endeavor. As such, myriad different factors can influence the outcome. Investigations of program comprehension, and in particular those using controlled experiments, have to take these factors into account. In order to promote the development and use of sound experimental methodology, we discuss potential problems with regard to the experimental subjects, the code they work on, the tasks they are asked to perform, and the metrics for their performance.

Original language	English
Title of host publication	Proceedings - 2021 IEEE/ACM 29th International Conference on Program Comprehension, ICPC 2021
Publisher	IEEE Computer Society
Pages	106-117
Number of pages	12
ISBN (Electronic)	9781665414036
DOIs	https://doi.org/10.1109/ICPC52881.2021.00019
State	Published - May 2021
Event	29th IEEE/ACM International Conference on Program Comprehension, ICPC 2021 - Virtual, Online Duration: 20 May 2021 → 21 May 2021

Publication series

Name	IEEE International Conference on Program Comprehension
Volume	2021-May

Conference

Conference	29th IEEE/ACM International Conference on Program Comprehension, ICPC 2021
City	Virtual, Online
Period	20/05/21 → 21/05/21

Bibliographical note

Publisher Copyright:
© 2021 IEEE.

Keywords

Code comprehension
experimental methodology

Access to Document

10.1109/ICPC52881.2021.00019

Cite this

Feitelson, D. G. (2021). Considerations and Pitfalls in Controlled Experiments on Code Comprehension. In Proceedings - 2021 IEEE/ACM 29th International Conference on Program Comprehension, ICPC 2021 (pp. 106-117). Article 9463032 (IEEE International Conference on Program Comprehension; Vol. 2021-May). IEEE Computer Society. https://doi.org/10.1109/ICPC52881.2021.00019

@inproceedings{36a951bdc4dc45818bdc58235619f75a,

title = "Considerations and Pitfalls in Controlled Experiments on Code Comprehension",

abstract = "Understanding program code is a complicated endeavor. As such, myriad different factors can influence the outcome. Investigations of program comprehension, and in particular those using controlled experiments, have to take these factors into account. In order to promote the development and use of sound experimental methodology, we discuss potential problems with regard to the experimental subjects, the code they work on, the tasks they are asked to perform, and the metrics for their performance.",

keywords = "Code comprehension, experimental methodology",

author = "Feitelson, {Dror G.}",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 29th IEEE/ACM International Conference on Program Comprehension, ICPC 2021 ; Conference date: 20-05-2021 Through 21-05-2021",

year = "2021",

month = may,

doi = "10.1109/ICPC52881.2021.00019",

language = "אנגלית",

series = "IEEE International Conference on Program Comprehension",

publisher = "IEEE Computer Society",

pages = "106--117",

booktitle = "Proceedings - 2021 IEEE/ACM 29th International Conference on Program Comprehension, ICPC 2021",

address = "ארצות הברית",

}

Feitelson, DG 2021, Considerations and Pitfalls in Controlled Experiments on Code Comprehension. in Proceedings - 2021 IEEE/ACM 29th International Conference on Program Comprehension, ICPC 2021., 9463032, IEEE International Conference on Program Comprehension, vol. 2021-May, IEEE Computer Society, pp. 106-117, 29th IEEE/ACM International Conference on Program Comprehension, ICPC 2021, Virtual, Online, 20/05/21. https://doi.org/10.1109/ICPC52881.2021.00019

Considerations and Pitfalls in Controlled Experiments on Code Comprehension. / Feitelson, Dror G.
Proceedings - 2021 IEEE/ACM 29th International Conference on Program Comprehension, ICPC 2021. IEEE Computer Society, 2021. p. 106-117 9463032 (IEEE International Conference on Program Comprehension; Vol. 2021-May).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Considerations and Pitfalls in Controlled Experiments on Code Comprehension

AU - Feitelson, Dror G.

PY - 2021/5

Y1 - 2021/5

N2 - Understanding program code is a complicated endeavor. As such, myriad different factors can influence the outcome. Investigations of program comprehension, and in particular those using controlled experiments, have to take these factors into account. In order to promote the development and use of sound experimental methodology, we discuss potential problems with regard to the experimental subjects, the code they work on, the tasks they are asked to perform, and the metrics for their performance.

AB - Understanding program code is a complicated endeavor. As such, myriad different factors can influence the outcome. Investigations of program comprehension, and in particular those using controlled experiments, have to take these factors into account. In order to promote the development and use of sound experimental methodology, we discuss potential problems with regard to the experimental subjects, the code they work on, the tasks they are asked to perform, and the metrics for their performance.

KW - Code comprehension

KW - experimental methodology

UR - http://www.scopus.com/inward/record.url?scp=85113240105&partnerID=8YFLogxK

U2 - 10.1109/ICPC52881.2021.00019

DO - 10.1109/ICPC52881.2021.00019

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85113240105

T3 - IEEE International Conference on Program Comprehension

SP - 106

EP - 117

BT - Proceedings - 2021 IEEE/ACM 29th International Conference on Program Comprehension, ICPC 2021

PB - IEEE Computer Society

T2 - 29th IEEE/ACM International Conference on Program Comprehension, ICPC 2021

Y2 - 20 May 2021 through 21 May 2021

ER -

Considerations and Pitfalls in Controlled Experiments on Code Comprehension

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this