Advancing EMS Through Research: Volume III
The "Advancing EMS Through Research" series highlights EMS eduational research projects conducted at our annual Fisdap Research Summit. To view more research projects, please visit: http://www.fisdap.net/research/projects.
Early Exposure to Valid Test Items Improves EMT Student Performance on Summative Exams
Ron Lawler, BUS, NRP; Elias Frick, BA, NRP; Ben Getsug, AAS; Brian Hendrickson, MS, EMT-P; Sara Houston, BS, NREMT-P; Evelyn Hunter, BS, MSW, LSW; Andy Lovell, NREMT-P; Luke Stanke, PhDc; Hanorah Vanni, BA; Mark Volker; Rachel Wasilewski, BS; Virginia Wilson, BSBA, BS, EMT-P
Introduction
There are several avenues for formative testing available to EMT programs, however most are not validated. Research in a variety of healthcare educational disciplines shows that valid formative examinations improve student outcomes on summative exams. Existing EMS-specific research is scarce on this subject.
Hypothesis
Exposure to valid, formative, secure EMT exams is predictive of success on valid, summative EMT exams.
Methods
EMT Readiness Exam 2 (ERE2) and EMT Unit Exam (EUE) results were retrospectively collected from Fisdap, an online EMS education data collection system from December 2010 through December 2014. Inclusion criteria consisted of student consent to research. Data points for the ERE2 and each EUE included scores and program ID. Data was fit to a hierarchical linear model (HLM) where level-1 contains information about the number of unit tests taken by a student and level-2 intercept for program affiliation (Table 1).
Table 1
Limitations
There are a number of unknown variables that this study does not control for including: context of EUE use (as pre-tests, module end exams, or pre-final exam prep), if exams were included in grade calculation and if there was any student exposure to other valid formative exams prior to the ERE2.
Results
The mean score of those students who took no EUEs was 67% (n = 22,359) while those who took all six exams was 71% (n = 1,557). Using the HLM to control for students in the same program, students who took all 6 unit exams still scored 1.8% higher on the ERE2 (Table 2).
Table 2
Next Steps
There is plenty of opportunity for further research in this area. Possible topics of research include: an examination of other valid test items used in place of/in addition to Fisdap Unit Exams, whether the length of program also has an effect on ERE success, whether Unit Exam exposure predicts success at the paramedic level, whether the timing of formative exam administration has an effect on the summative exam results, whether a student’s score on the first formative exam is predictive of attrition, and whether the administration of a valid entrance examination would positively impact summative exam results. There’s also an opportunity to examine whether exposure to formative exams specifically impacts performance on the critical thinking portion of the summative exams.