Evaluation Foundations
Who are the individuals or groups interested in or affected by a program evaluation?
Stakeholders
A survey recruits people who happen to be at the agency on Tuesday morning. Name the sampling approach.
Convenience sampling
An agency wants to know whether clients feel respected. Whose perspective is essential to collect?
Client/participant feedback.
What type of evaluation helps develop or improve a program before full implementation?
Formative evaluation
What tool illustrates how program activities are expected to lead to results?
Logic Model
What is the primary purpose of conducting a program evaluation?
To assess and improve a program or inform decisions
Only program completers are surveyed about satisfaction. Name one likely bias.
Selection or nonresponse bias; noncompleters are excluded.
A survey question asks whether staff were respectful AND helpful. What is wrong with the item?
It is double-barreled.
What type examines whether program services are delivered as intended?
Process evaluation
Staff, funding, and facilities are examples of what component?
Inputs
What do we call the specific questions an evaluation is designed to answer?
Evaluation questions
An evaluation collects surveys and interviews. What broad methodological approach is being used?
Mixed methods
A measure gives consistent results but fails to capture the intended concept. What measurement property is weak?
Validity
What type examines whether participants achieve intended changes?
Outcome evaluation
Providing workshops, counseling, and outreach are examples of what component?
Activities
An agency wants to know whether families maintained housing for 12 months. Is this primarily a process or an outcome question?
Outcome question
Scores improve after a program, but there is no comparison group. Name one major interpretation concern.
Other factors, maturation, history, or regression to the mean may explain change; causality is limited.
A youth program counts attendance as a measure of belonging. What is the key concern?
Attendance is not a direct measure of belonging.
What evaluation identifies gaps between current and desired conditions?
Needs assessment
Increased knowledge, improved skills, and changes in behavior are examples of what component?
Outcomes
Name one reason to involve participants in deciding what a program evaluation should measure.
Participant priorities, experiences, or definitions of meaningful outcomes.
An evaluator wants to understand why some participants leave a program before completing it. What data collection approach would help them better understand participants' experiences?
Interviews with participants who discontinued services.
An agency conducts interviews with 20 program participants. What is one way the evaluator could systematically analyze the information collected?
Qualitative coding or thematic analysis to identify patterns or themes across interviews.
An agency wants to examine both how consistently services are delivered and whether participants experience meaningful improvements. What two types of evaluation would be appropriate?
Process evaluation and outcome evaluation.
A community program plans to expand its services but discovers it does not have enough staff or funding. Which part of its logic model needs to be reconsidered?
Inputs