Performance assessment of community-based physicians: Evaluating the reliability and validity of a tool for determining CME needs

Robert Cohen*, Gilad E. Amiel, Mark Tann, Aric Shechter, Michael Weingarten, Shmuel Reis

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

12 Scopus citations

Abstract

Purpose. To evaluate the reliability, validity, and feasibility of the Physician Assessment in Medical Practice (PAMP) as a means of determining the CME needs of practicing, community-based physicians. Method. A group of 45 randomly selected community- based physicians (19 certified family physicians and 26 general practitioners) affiliated with the Department of Family Medicine at Bruce Rappaport Faculty of Medicine, Technion Institute of Technology, Haifa, Israel, volunteered to participate in the study, conducted in 1997. All participants took a ten-station, performance-based examination designed to closely represent the physicians' work settings. At each station, a different medical problem was presented by a standardized patient. Physician - candidates' performances were assessed by physician - examiners using global ratings. The following performance domains were assessed: information gathering, diagnosis and management plan, and communication skills. A CME needs assessment score was determined for each of the participants and a CME level to meet the needs of the physician was recommended. Results. Overall reliability of the examination was high (.87), with domain reliabilities ranging from .76 to .87. Reliability of the examiners' judgments of the physicians' competence was .66. All the stations' validity scores were significant, and differences in performances between family physicians and general practitioners demonstrated construct validity of the test results. Overall, the cost of running the examination was U.S. $250 per physician -candidate. Conclusions. Using the PAMP to determine CME needs of community-based physicians was found to reliable, valid, and feasible, and the cost per physician - candidate was not excessive. Performance results provided in-depth information for use by both the individual physician and providers of CME programs.

Original languageAmerican English
Pages (from-to)1247-1254
Number of pages8
JournalAcademic Medicine
Volume77
Issue number12 I
DOIs
StatePublished - 1 Dec 2002
Externally publishedYes

Fingerprint

Dive into the research topics of 'Performance assessment of community-based physicians: Evaluating the reliability and validity of a tool for determining CME needs'. Together they form a unique fingerprint.

Cite this