Evaluation Process
This step in the evaluation process entails identifying what knowledge, skills, behaviors, or other learned capabilities are needed. (Identifying the gap!)
Conducting a needs analysis
Have participants applied what they learned from the training?
This represents what level of outcomes in Kirkpatrick's Model? (need number)
Level 3
Group of employees who participate in the evaluation study but do not attend the training program
Comparison Group
True or False: There is no one appropriate evaluation design that should be used in every circumstance.
True
Name one outcome you might assess for a training on bus driver safety.
Some examples might be: trainees’ satisfaction with the course, knowledge about how to operate the bus, safe driving behaviors, and accident rates
This part of the evaluation process may be considered the "dress rehearsal." The results are used to modify, market, or gain additional support for the program.
Formative evaluation
Indicate the extent to which you agree with the following: "The materials I received during the course were useful."
The above item represents an example of this type of training outcome.
Reaction
Because it is often impossible to identify and measure all the individual characteristics that might influence the training outcome measures, _________ ensures that these characteristics are equally distributed in the comparison group and the training group.
random assignment
In most cases, reaction outcomes are measured how long after the training has been delivered?
Immediately after training
According to the last issue of I-O Kiddo's (monthly newsletter created by the students for the students), Dr. O wants to fly with this type of bird.
Turkeys
Step 2 of the evaluation process entails developing learning objectives to guide the program. These training objectives should have these TWO characteristics to make it easier to identify relevant outcomes for the evaluation.
Specific & Measurable
Level 2 outcomes (learning or cognitive) are typically measured with one of these two methods.
Tests and Work Samples
The believability of your training evaluation results speaks to this kind of validity.
Internal validity
Companies should select training outcomes related to these TWO things.
Learning and Transfer of Training
According to Kirkpatrick's Model, outcomes that assess payoffs for the company and answer the question "Did it impact the bottom line?" are called ______.
Results
Step 3 of the evaluation process entails developing _______ _______. You'll want these outcome measures to assess the extent to which learning and transfer have occurred.
outcome measures
Tolerance for diversity, safety attitudes, and engagement are all examples of this type of outcome.
Affective
The main advantage of ______ evaluation design is that it allows an analysis of the stability of training outcomes over time.
Time Series
When evaluating training results, it is considered best practice to explore extraneous factors that could have impacted how effective the training appeared to be.
Consider a group of managers who have attended a communication skills training program. At the same time that they attend the program, it is announced that the company will be restructured. After the program, the managers may become better communicators simply because they are scared that if they don’t they will lose their jobs. Perhaps no learning actually occurred in the training program at all.
This type of threat to internal validity is called company ______.
history
When conducting a cost-benefit analysis, what are the two categories of cost sources to consider when conducting an ROI analysis?
Direct & Indirect
A summative evaluation usually centers on what kind of data?
quantitative
ROI is calculated using this formula.
[(Benefit of training)-(Training Costs)] / (Training Costs) x 100%
A post-test-only with no comparison group is considered low in strength for establishing validity. When is the only time using this evaluation design is considered appropriate?
When trainees can be expected to have similar levels of knowledge, behavior, or results outcomes prior to training.
At what point should learning, behavior, and results outcomes be measured?
After sufficient time has elapsed to determine whether training has had an influence on these outcomes.
What song, written by the Bee Gees, made an appearance in a video example during a previous class?
Stayin' Alive