Validity and generalizability of global ratings in an objective structured clinical examination

The performance of foreign medical graduates on multistation standardized patient-based tests was used to determine the validity and generalizability of global ratings of their clinical competence made by expert examiners. Data were derived from the entrance examinations of the 1989 and 1990 applicants to the Ontario Pre-Internship Program and the exit examination of 24 participants from the 1989 cohort. For each candidate, the examiners completed a detailed checklist and two five-point global ratings dealing with the candidate's approach to the patient's problem and attitude toward the patient. Generalizability coefficients for both ratings were satisfactory and stable across cohorts. Construct validity of the global ratings was demonstrated by comparing entry and exit ratings and by evidence of significant and positive correlations between the global ratings and total test scores. Tentative evidence of criterion validity of the global ratings was demonstrated. These findings suggest that global ratings by expert examiners can be used as an effective form of assessment in multistation standardized patient examinations.