Table 1 RE-AIM evaluation plan to evaluate HIRAID® implementation strategy
RE-AIM domain and definition | Intended outcome | Measure / Indicator | Data Source |
|---|---|---|---|
Reach The number, proportion, and representativeness of individuals willing to participate in the HIRAID® program [29] | Individual: > 80% of nurses from all 29 EDs complete HIRAID® training program representing novice to experienced nurses from all ED types | % of eligible nurses that completed (i) any training (ii) all training (iii) % sites represented % of eligible nurses that (iv) completed post intervention survey (v) % sites represented (vi) range of years of experience of those attending | Study records Staff survey |
Setting: 100% sites have a senior nurse “Instructor” complete HIRAID® Instructor program | Proportion of eligible sites that had a nurse complete instructor training | Study records | |
Effectiveness* Effectiveness assesses whether the targeted behavioural outcome was achieved and changes to quality of life (QOL) or other important outcomes [1] *Patient, health service and costing effectiveness to be reported elsewhere. Purpose of this paper is to evaluate implementation strategy on implementation | > 80% nurse participant opinion on the usefulness of HIRAID® Sub-group analyses to draw conclusions about how different subpopulations responded to the intervention Percent attrition Results for at least one follow-up | % eligible nurses who indicated HIRAID® helps: (i) remember all parts of a full assessment (ii) teach new emergency nurses how to assess and manage patients | Staff survey + free text comments |
Uptake of BCTTs – opinion from staff on particular Behaviour change techniques (BCTs) | Survey Free text | ||
Adoption (settings level) The number of sites eligible and willing to initiate a program, and why (Individual level) Delivery staff invited to participate, and characteristics of participants) and intervention adoption rate | 100% of eligible sites implementing HIRAID® Compare their characteristics to nonparticipating settings (N/A) | % of eligible EDs that implement HIRAID® | Study records |
100% Senior nurse “Instructor” from each site complete HIRAID® Instructor program | % of sites that have staff attend instructor course | Study records | |
> 80% use of HIRAID® templates to document patient assessment | % patients who had HIRAID® documentation when needed | Audits uptake 6 weeks, 12 weeks Staff survey Free text comments explaining why or why not used | |
Implementation quality (dose and fidelity) Consistency of implementation across settings and within planned time frames to the intervention strategy, including time and cost as intended | Dose: > 90% of 21 BCTTs used [30] > 80% within timeframe | Proportion of BCTTs implemented by total number planned - Number of instructor and provider courses delivered at how many sites to how many nurses - eLearning % completions - Face to face sessions % completions - Number face to face sessions - Number champions at each site - Time frame delivered per protocol | Study records |
Fidelity: > 80% of perfect delivery by the HIRAID® team at (i) Instructor course (setting) (ii) Provider course (individual). Did each Instructor deliver the same material? < 30% adaptations made to BCTs and implementation strategies and reasons | % of elements delivered in Instructor program and materials delivered % of 15 prescribed elements delivered all the time, most of the time, hardly ever % adaptations made to HIRAID® delivery or implementation strategy (quant and qual) % of elements used (doc templates, memo, flip card etc.) | Instructor course delivery Study 15 item Instructor survey (quant) Implementation logs (qual) HIRAID® Instructors completed an intervention fidelity scoring sheet that asks to what degree the behaviour change mechanisms were implemented [22] | |
Quality: Level of staff satisfaction with the delivery of education related to the HIRAID® framework | The proportion of staff surveyed that rated the education to be delivered in an engaging manner | Staff survey | |
Cost of Intervention Measures of cost of implementation | Cost of face-to-face attendance by staff. Any additional resources needed or done within budget | Research partner letters | |
Maintenance / Sustainability The maintenance dimension assesses both individual-level behaviour change and organizational-level intervention sustainability [30] | > 90% sites have HIRAID® embedded business as usual ≥ 6 month post study (Alignment with organizational mission) > 80% participants state they are using ≥ 6 months post intervention Qualitative measure of individual-level maintenance | Proportion of sites with HIRAID® education program embedded in new nurse orientation program and how (which elements retained) Proportion of sites with HIRAID® documentation templates in use Proportion of nurses saying they are using it 6 months post implementation | Instructor survey Local audit 2yrs post implementation Nursing survey |