Name that Evaluation
Evaluation occurs after an educational activity is completed and provides feedback about the activity's effectiveness in achieving the desired outcome
Summative evaluation
The learner will demonstrate engagement as evidence by asking questions and participating in the activity.
Level 1 Evaluation: Reaction
The action verb "identify" is best matched with what test item?
Multiple-choice or matching question
Expenses of program design and implementation is reported as what component of ROI?
Total Costs
The staff request education on drain management.
What would be the recommended evaluation of this learning activity?
Level 2 Evaluation: Learning
Pre-/post-tests
Demonstration
Group activity - game, oral Q&A
Any measurement of knowledge gained
Evaluation occurs during the educational activity and allows the learning facilitator to adjust content and/or learner engagement methods to meet the needs of participants
Formative evaluation
The aim of this educational activity is to evaluate the effectiveness of game-based learning in reducing central line blood stream infections, catheter associated urinary tract infections, and fall rates at 3 months, 6 months, and 12 months post-learning activity.
Level 4 Evaluation: Results
The action verb "explain" is best matched with what test item?
Short answer or essay question
Revenue, positive outcomes, and savings from avoidance of poor outcomes is reported as what component of ROI?
Total Benefits
The nurse manager contacts you to develop and implement education on early signs of patient deterioration.
What would be the recommended evaluation of this learning activity?
Level 2 Evaluation: Learning
Simulation
Pre-/Post-test
Case studies
Evaluation assesses learners' knowledge or skills before beginning a learning activity
Diagnostic evaluation
In accordance with End-of-Life Nursing Education Consortium (ELNEC) guidelines, the medical-surgical nurse will develop a care plan for a terminally ill patient that includes strategies for meeting the emotional needs of both the patient and family.
Level 2 Evaluation: Learning
The action verb "state" is best matched with what test item?
Short answer or sentence-completion question
What is the formula for calculating ROI?
(Total Benefit - Total Cost of Program)/Total Cost of Program x 100 = ____% ROI
Nurse retention rates are low, and you know that it is the organization's goal to ensure staff wellbeing. You plan to implement a wellbeing program to address this issue.
What would be the recommended evaluation of this learning activity?
Level 3 Evaluation: Behavior
Surveys
Discussion groups
Follow-up with learners
Supportive data (e.g., lunches not missed, call outs, late arrival, teammate recognition)
Evaluation compares an individual's performance to established competency standards, such as competencies stated in a job description or in standards of practice
Competency-based evaluation
The clinician will implement a self-developed stress reduction plan using the guidelines from "Stress Reduction Strategies".
Level 3 Evaluation: Behavior
The action verb "interpret" is best matched with what test item?
Exercise to interpret cardiac rhythm strip, lab values, table, or other graphic
What programs are recommended for ROI?
High visibility, regulatory compliance - e.g., sepsis, heart failure, stroke
High importance of administrators or error prevention - e.g., patient safety events
Alignment with mission & strategic plan- e.g., DE&I, zero patient harm
Far-reaching impact or behaviors that are high litigation or high cost to the organization- e.g., CAUTI, CLABSI
Your hospital system wants to make significant changes to the orientation program including the transition of some content to a new virtual platform, reducing the number of in-person hours for orientation.
What would be the recommended evaluation of this learning activity?
Return on Investment
Evaluation compares learners' performance with criteria established in advance (not comparing to other learners' performance)
Criterion-referenced evaluation
The implementation of a structured preceptor development program will increase retention of nurses at a community-based hospital.
Return on Investment
The action verb "differentiate" is best matched with what test item?
Multiple choice, short answer, or essay question depending on the context
What is the return on investment for changing a course on nursing care for patients with heart failure from in-person to virtual?
The expenses of the in-person class include:
- Door prizes (2): $25 per item
- Handouts (30): $5 total for printing
- Facility costs (rental): $200
- NPD practitioner wages: NPD practitioner has an hourly wage of $51. You need two NPD practitioners to plan (about 2h planning) and teach the class (4h class).
The expenses of the virtual class include:
- NPD practitioner wages: NPD practitioner has an hourly wage of $51. You need one NPD practitioner to develop and implement this virtual training (about 7 hours total).
Total Cost of In-person: $1,012
Total Cost of Virtual: $357
Overall Cost Difference: $655
($1,012 - $357) / $357 x 100 = 183% ROI
The department's patient satisfaction scores are low related to the question, "During this hospital stay, how often did nurses explain things in a way you could understand?". Key stakeholders develop a team to address this issue. You are asked as the NPD practitioner to create education (e.g., bedside shift report, teach-back) to address this concern.
What would be the recommended evaluation of this learning activity?
Level 4 Evaluation: Results
Patient Satisfaction Scores (pre/post)
Other supportive data