A fresh look at novice programmers' performance and their teachers' expectations

Ian Utting; Allison Elliott Tew; Mike McCracken; Lynda Thomas; Dennis Bouvier; Roger Frye; James Paterson; Michael Caspersen; Yifat Ben David Kolikant; Juha Sorva; Tadeusz Wilusz

doi:10.1145/2543882.2543884

A fresh look at novice programmers' performance and their teachers' expectations

Ian Utting, Allison Elliott Tew, Mike McCracken, Lynda Thomas, Dennis Bouvier, Roger Frye, James Paterson, Michael Caspersen, Yifat Ben David Kolikant, Juha Sorva, Tadeusz Wilusz

Department of Education

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

40 Scopus citations

Abstract

This paper describes the results of an ITiCSE working group convened in 2013 to review and revisit the influential ITiCSE 2001 McCracken working group that reported [18] on novice programmers' ability to solve a specified programming problem. Like that study, the one described here asked students to implement a simple program. Unlike the original study, students' in this study were given significant scaffolding for their efforts, including a test harness. Their knowledge of programming concepts was also assessed via a standard language-neutral survey. One of the significant findings of the original working group was that students were less successful at the programming task than their teachers expected, so in this study teachers' expectations were explicitly gathered and matched with students' performance. This study found a significant correlation between students' performance in the practical task and the survey, and a significant effect on performance in the practical task attributable to the use of the test harness. The study also found a much better correlation between teachers' expectations of their students' performance than in the 2001 working group.

Original language	American English
Title of host publication	ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education
Publisher	Association for Computing Machinery
Pages	15-32
Number of pages	18
ISBN (Print)	9781450326650
DOIs	https://doi.org/10.1145/2543882.2543884
State	Published - 2013
Event	18th ACM Conference on Innovation and Technology in Computer Science Education, ITiCSE-WGR 2013 - Canterbury, United Kingdom Duration: 1 Jul 2013 → 3 Jul 2013

Publication series

Name	ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education

Conference

Conference	18th ACM Conference on Innovation and Technology in Computer Science Education, ITiCSE-WGR 2013
Country/Territory	United Kingdom
City	Canterbury
Period	1/07/13 → 3/07/13

Keywords

Assessment
CS1
Programming
Replication

Access to Document

10.1145/2543882.2543884

Cite this

Utting, I., Tew, A. E., McCracken, M., Thomas, L., Bouvier, D., Frye, R., Paterson, J., Caspersen, M., Kolikant, Y. B. D., Sorva, J., & Wilusz, T. (2013). A fresh look at novice programmers' performance and their teachers' expectations. In ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education (pp. 15-32). (ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education). Association for Computing Machinery. https://doi.org/10.1145/2543882.2543884

Utting, Ian ; Tew, Allison Elliott ; McCracken, Mike et al. / A fresh look at novice programmers' performance and their teachers' expectations. ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education. Association for Computing Machinery, 2013. pp. 15-32 (ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education).

@inproceedings{132e0809f90e4f3793ad085f57f86519,

title = "A fresh look at novice programmers' performance and their teachers' expectations",

abstract = "This paper describes the results of an ITiCSE working group convened in 2013 to review and revisit the influential ITiCSE 2001 McCracken working group that reported [18] on novice programmers' ability to solve a specified programming problem. Like that study, the one described here asked students to implement a simple program. Unlike the original study, students' in this study were given significant scaffolding for their efforts, including a test harness. Their knowledge of programming concepts was also assessed via a standard language-neutral survey. One of the significant findings of the original working group was that students were less successful at the programming task than their teachers expected, so in this study teachers' expectations were explicitly gathered and matched with students' performance. This study found a significant correlation between students' performance in the practical task and the survey, and a significant effect on performance in the practical task attributable to the use of the test harness. The study also found a much better correlation between teachers' expectations of their students' performance than in the 2001 working group.",

keywords = "Assessment, CS1, Programming, Replication",

author = "Ian Utting and Tew, {Allison Elliott} and Mike McCracken and Lynda Thomas and Dennis Bouvier and Roger Frye and James Paterson and Michael Caspersen and Kolikant, {Yifat Ben David} and Juha Sorva and Tadeusz Wilusz",

year = "2013",

doi = "10.1145/2543882.2543884",

language = "American English",

isbn = "9781450326650",

series = "ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education",

publisher = "Association for Computing Machinery",

pages = "15--32",

booktitle = "ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education",

note = "18th ACM Conference on Innovation and Technology in Computer Science Education, ITiCSE-WGR 2013 ; Conference date: 01-07-2013 Through 03-07-2013",

}

Utting, I, Tew, AE, McCracken, M, Thomas, L, Bouvier, D, Frye, R, Paterson, J, Caspersen, M, Kolikant, YBD, Sorva, J & Wilusz, T 2013, A fresh look at novice programmers' performance and their teachers' expectations. in ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education. ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education, Association for Computing Machinery, pp. 15-32, 18th ACM Conference on Innovation and Technology in Computer Science Education, ITiCSE-WGR 2013, Canterbury, United Kingdom, 1/07/13. https://doi.org/10.1145/2543882.2543884

A fresh look at novice programmers' performance and their teachers' expectations. / Utting, Ian; Tew, Allison Elliott; McCracken, Mike et al.
ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education. Association for Computing Machinery, 2013. p. 15-32 (ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A fresh look at novice programmers' performance and their teachers' expectations

AU - Utting, Ian

AU - Tew, Allison Elliott

AU - McCracken, Mike

AU - Thomas, Lynda

AU - Bouvier, Dennis

AU - Frye, Roger

AU - Paterson, James

AU - Caspersen, Michael

AU - Kolikant, Yifat Ben David

AU - Sorva, Juha

AU - Wilusz, Tadeusz

PY - 2013

Y1 - 2013

N2 - This paper describes the results of an ITiCSE working group convened in 2013 to review and revisit the influential ITiCSE 2001 McCracken working group that reported [18] on novice programmers' ability to solve a specified programming problem. Like that study, the one described here asked students to implement a simple program. Unlike the original study, students' in this study were given significant scaffolding for their efforts, including a test harness. Their knowledge of programming concepts was also assessed via a standard language-neutral survey. One of the significant findings of the original working group was that students were less successful at the programming task than their teachers expected, so in this study teachers' expectations were explicitly gathered and matched with students' performance. This study found a significant correlation between students' performance in the practical task and the survey, and a significant effect on performance in the practical task attributable to the use of the test harness. The study also found a much better correlation between teachers' expectations of their students' performance than in the 2001 working group.

AB - This paper describes the results of an ITiCSE working group convened in 2013 to review and revisit the influential ITiCSE 2001 McCracken working group that reported [18] on novice programmers' ability to solve a specified programming problem. Like that study, the one described here asked students to implement a simple program. Unlike the original study, students' in this study were given significant scaffolding for their efforts, including a test harness. Their knowledge of programming concepts was also assessed via a standard language-neutral survey. One of the significant findings of the original working group was that students were less successful at the programming task than their teachers expected, so in this study teachers' expectations were explicitly gathered and matched with students' performance. This study found a significant correlation between students' performance in the practical task and the survey, and a significant effect on performance in the practical task attributable to the use of the test harness. The study also found a much better correlation between teachers' expectations of their students' performance than in the 2001 working group.

KW - Assessment

KW - CS1

KW - Programming

KW - Replication

UR - http://www.scopus.com/inward/record.url?scp=84893416747&partnerID=8YFLogxK

U2 - 10.1145/2543882.2543884

DO - 10.1145/2543882.2543884

M3 - Conference contribution

AN - SCOPUS:84893416747

SN - 9781450326650

T3 - ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education

SP - 15

EP - 32

BT - ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education

PB - Association for Computing Machinery

T2 - 18th ACM Conference on Innovation and Technology in Computer Science Education, ITiCSE-WGR 2013

Y2 - 1 July 2013 through 3 July 2013

ER -

Utting I, Tew AE, McCracken M, Thomas L, Bouvier D, Frye R et al. A fresh look at novice programmers' performance and their teachers' expectations. In ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education. Association for Computing Machinery. 2013. p. 15-32. (ITiCSE-WGR 2013 - Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education). doi: 10.1145/2543882.2543884

A fresh look at novice programmers' performance and their teachers' expectations

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this