We need this, stat!
The analytic strategy you should employ if there are differences in the outcome at pre-test between two groups
What is change in change scores, or difference in differences?
Program exposure is considered this type of variable (dependent or independent)
What is an Independent Variable?
The threat that exists when control group members participated in the intervention
What is contamination?
When evaluators assign a group of study participants to receive the intervention or not receive the intervention randomly
What is random allocation or random assignment?
A type of evaluation where you assess fidelity of program implementation.
What is process evaluation?
The statistical test used to compare the proportion of outcomes in two groups
Chi2 test
Variables that are thought or known to influence program outcomes, such as participant characteristics
What are moderators?
For example, when a pre-test was administered on paper and the post-test was administered via computer
What is instrumentation?
When evaluators use a sampling frame from a population and pick subjects randomly to be in the study
This involves testing a causal relationship between a program and the anticipated results
What is outcome evaluation?
The statistical test that could be used in a post-only evaluation to measure changes between program groups in days since last use of opioids (for a relapse/overdose prevention program)
What is a t-test?
What would have what would have happened if the intervention had not taken place – a critical piece in impact evaluation
What is a counterfactual?
Participants who are most likely to benefit from a program drop out of the program due to time constraints, while those in the control group continue to participate
What is attrition?
The process in an evaluation study that helps to control for known and unknown confounders
What is randomization? or Random allocation?
This involves testing a causal relationship between a program and hypothesized results that can be attributable to the program using statistical method
What is impact evaluation?
This test would be used when comparing individuals to themselves before and after a program, using a binary/nominal outcome expressed as proportions (such as % who quit smoking)
What is a McNemar's test?
The type of validity that is how well indicators describe the constructs and causal linkages in the evaluation model
What is construct validity?
For example, A TikTok video goes of an adolescent celebrity/influencer making remarks about how they recently quit smoking after 15 years and feel amazing right in the middle of a smoking cessation evaluation among teens
What is history?
Evaluators often construct a "Table 1" comparing intervention and control in a study population to confirm the success or failure in this process.
What is randomization?
The type of evaluation that is conducted before implementation to understand how the program should implemented
What is Formative evaluation?
The test you would use when comparing individuals to themselves before and after a program, using a continuous variable such as mean number of pounds lost
Paired t-test
The type of validity concerning when assumptions that project interventions have caused observed outcomes may not be valid
What is internal validity?
The evaluation team selects a group to intervene with who have extreme behaviors, and use a control group that do no have these behaviors at the start of the study
What is regression?
The type of error that occurs when there is not enough power to detect a difference that actually exists
What is Type 2 error?
The type of evaluation that involves assessing dose delivered, dose received and reach, for example.
What is process evaluation?