The following abstract was developed during the 2013 Research Summit and presented at the 2014 NAEMSP Annual Meeting in Tuscon, AZ.

Student Self-Evaluation: Learning to "Know Thyself"

James D. Dinsch, MS, NREMT-P; Greg Cliburn, MA, NRP; William Gaskill; Liz Mrak, BS, NREMT-P; R. Adam Parker, AAS, NREMT-P, FP-C; Sara Richter, MS, EMT-B; Dana Snuggerud, BAS; Rachel Wasilewski, BS; Todd M. Cage, M.Ed., NREMT-P

Introduction: The ability to evaluate one’s own performance is an important professional quality and a goal of paramedic affective education. Paramedic students are often required to evaluate their own performance, but how well they self-evaluate, how they learn to improve their self-evaluation skills, and the effect of these skills on their overall academic performance have not been studied. The purpose of this study was to determine whether the accuracy of student self-evaluation improves over time and whether or not that has an effect on academic performance.

Hypotheses: (a) By the completion of the field experience, paramedic student selfevaluation aligns with preceptor evaluation of the student. (b) Students who are accurate self-evaluators of their field experience perform better on a summative cognitive examination.

Methods: Three paramedic programs were identified as having participated in the National Registry of Emergency Medical Technician’s Paramedic Psychomotor
Competency Portfolio Package Project and used FISDAP®, a national, online EMS student tracking system. Field experience and comprehensive paramedic exam performance data were abstracted from the FISDAP® database for students who had “good” data and consented to have their data used for research. Student and preceptor evaluations were compared for seven categories (Communication, Field Impression, Interview, Physical Examination, Professional Behavior, Skills Performed, and Team Leadership) using Cohen’s weighted kappa coefficient. Evaluation agreement over time was evaluated using a Chi-square and relationship to exam pass/fail status examined with the Mann-Whitney-Wilcoxon test.

Results: The dataset included 34,570 evaluated field patient encounters for 267 students. The percent of encounters in which both preceptor and student completed the evaluation was high (76.25-82.11%) as was overall agreement in all seven categories (κ=0.8106-0.8448). Student awareness of when to rate and preceptor-student agreement across all categories increased with increased experiences (p<0.0001). Higher agreement relative to exam passing was only significant for Professional Behavior (p=0.0076).

Discussion: Student awareness of when to rate and agreement with preceptors’ ratings were high overall, but increased exposure did increase the level of agreement. While agreement relative to exam passing status was significant for Professional Behavior, overall high agreement in both pass and fail groups across all categories offers no practical importance.