Basic
The 3 types of Evaluation
Formative, Process, and Summative
An Example of History Bias
Exposure to previous intervention, political or social event
The gold standard
Randomized Control Trial
The Formula for Objectives
Specific
Measurable
Attainable
Realistic
Timebound
Recruitment, Refusal, Retention, Attrition
Who?
How Many?
For How Long?
Social Desirability, Recall Bias, Interviewer Bias
A reflexive control design
one group compared against itself, weak design for internal validity, pretest serves as counterfactual
Characteristics of a Health Goal
Defines the health change, Indicates a direction (increase or decrease), Specifies target group, Specifies location
2 Common Problems with Survey Questions
too long
complex words
leading questions
inappropriate reference frame
double barreled
using absolutes
Characteristics of Internal Evaluators
more knowledgeable about program and context
less objectivity
more community trust
cheaper
Randomize or develop solid matching criteria with a sufficient sample
Don't allow self-assignment
Ensure a complete sampling frame
The 3 Steps of Matching
1. Identify Targets for Intervention
2. Select relevant criteria
3. Build a control group
The parts of a LM
Inputs, Activities, Outputs, Outcomes, Health Goal, External Factors
An indicator of dose delivered (numerator and denominator)
% of training sessions provided
% of participants who received training materials
% of sessions with demonstration exercises implemented
etc
2 Theories of Behavior Change
Health Belief Model, Social Cognitive Theory, Self-Efficacy Theory, etc
An example of maturation bias
Teens aging, and becoming better educated
Reasons not to utilize RCT design
The specific uses of formative evaluation
Evaluate nature of the problem
Help determine target population (and recruitment/retention)
Determine measurement procedure
Ensure intervention is culturally appropriate
Pre-test instruments and materials
Understand why a program does or doesn’t work
The 3 Groups of Matching Variables
1. Composition
2. Predisposition
3. Experience
increase buy in, foster trust and facilitate evaluation process, foster strategic thinking, impact quality/use of findings, increase evaluation capacity
Threats to Validity During the Design Phase
Altering TOC, altering quantity or quality of activities, inappropriate target audience, etc
The notation for a matched quasi-experimental delay design
O1e - > Xe -> O2e -> O3e
(Group) -> Matched
O1c - > O2c -> Xc -> O3c
Factors of Evaluability Assessment
Stratified Sampling
The population is separated into subgroups to obtain samples.