Sampling
Study Designs
Internal Validity
Survey design
Evaluator terminology
100

Participants share similar key characteristics, such as age, gender, and motivation. What kind of sampling is this?

Matching pairs sampling

100

What challenges are created when using a pre-post study design?

Confounding issues arise; history and maturation bias are a concern; no ability to prove causation

100

Recruitment involved selecting fifty individuals aged 18-40 on a first-come, first-served basis. What does this mean for internal validity?

Self-selection bias is introduced, and there may not be equal demographics included e.g., more women than men. More motivated participants will self-select into the study. 

100

A double-barreled question does what?

Asks about two distinct topics in one question. 

100

Researchers use this experimental technique to combat order effects by varying the sequence in which participants experience different conditions.

Counterbalancing

200

Participants are asked to recruit future participants for a study. What kind of sampling is this?

Snowball sampling

200

A design that measures a single group of participants before and after an intervention but lacks a control group.

One-group pretest-posttest design

200

Time-based changes in participants—such as biological growth, aging, gains in knowledge, and cognitive development are what kind of threat to internal validity?

It is maturation bias, which may skew the results as the evaluator attributes the change in outcome to the program or intervention, when it might just be time. 

200

A type of questionnaire item that sneaks in an assumption or pushes a respondent toward a predetermined answer

Leading question

200

This threat to internal validity occurs when the control group inadvertently receives the intervention meant only for the treatment group.

Contamination

300

What are the benefits of a random sample?

It addresses confounding, allows us to detect cause and effect relationships, and improves representativeness, and reduces selection bias.

300

What kind of design measures a dependent variable repeatedly before, during, and after a specific interruption or treatment event?

Interrupted time series
300

When a localized program runs alongside a sweeping national policy change, what internal validity threat are we facing?

Confounding, there is an information effect of the national change that we need to address. 

300

Having participants take the same survey over time can introduce what kind of threat to internal validity?

Testing effect, they become comfortable or familiar with the survey and format. 

300

Refers to the degree to which a study can confidently establish a causal relationship between an independent and dependent variable, free from confounding factors.

Internal validity

400

This purposive sampling technique deliberately selects outliers

Case deviant sampling

400

A research design enhances ethical viability by ensuring that the initial control or comparison group is not permanently denied the beneficial treatment.

Switching Replications Design

400

Participant loss over time in a longitudinal study can introduce bias if those who drop out differ systematically from those who remain. What is this called?

Attrition

400

When designing a Likert scale, these two golden rules ensure the options mirror each other on both sides and are evenly spaced.

Symmetry and balance

400

If Alexander Fleming had properly cleaned his laboratory dishes before going on vacation in 1928, penicillin as an antibiotic might never have been discovered. What is this called?

Counterfactual: it is what would have happened had the event/interruption/intervention/program not taken place.